CN114071228A - Real-time information superposition method and device, electronic device and storage medium - Google Patents
Real-time information superposition method and device, electronic device and storage medium Download PDFInfo
- Publication number
- CN114071228A CN114071228A CN202111143264.9A CN202111143264A CN114071228A CN 114071228 A CN114071228 A CN 114071228A CN 202111143264 A CN202111143264 A CN 202111143264A CN 114071228 A CN114071228 A CN 114071228A
- Authority
- CN
- China
- Prior art keywords
- frame
- information
- canvas
- superposition
- real
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 60
- 238000004590 computer program Methods 0.000 claims description 14
- 238000012216 screening Methods 0.000 claims description 12
- 238000006243 chemical reaction Methods 0.000 claims description 10
- 238000009877 rendering Methods 0.000 claims description 3
- 230000004044 response Effects 0.000 claims description 3
- 238000012545 processing Methods 0.000 abstract description 15
- 230000008569 process Effects 0.000 abstract description 13
- 238000005516 engineering process Methods 0.000 abstract description 11
- 230000015572 biosynthetic process Effects 0.000 abstract description 10
- 238000003786 synthesis reaction Methods 0.000 abstract description 10
- 230000002194 synthesizing effect Effects 0.000 abstract description 9
- 230000003111 delayed effect Effects 0.000 description 17
- 230000005540 biological transmission Effects 0.000 description 7
- 238000010586 diagram Methods 0.000 description 7
- 230000003993 interaction Effects 0.000 description 4
- 238000004891 communication Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000002452 interceptive effect Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000013467 fragmentation Methods 0.000 description 1
- 238000006062 fragmentation reaction Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 230000000873 masking effect Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000035515 penetration Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
- H04N21/440218—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by transcoding between formats or standards, e.g. from MPEG-2 to MPEG-4
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
- H04N19/174—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a slice, e.g. a line of blocks or a group of blocks
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/21—Server components or server architectures
- H04N21/218—Source of audio or video content, e.g. local disk arrays
- H04N21/2187—Live feed
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/433—Content storage operation, e.g. storage operation in response to a pause request, caching operations
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/4405—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving video stream decryption
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/478—Supplemental services, e.g. displaying phone caller identification, shopping application
- H04N21/4788—Supplemental services, e.g. displaying phone caller identification, shopping application communicating with other users, e.g. chatting
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The application relates to a real-time information superposition method, a real-time information superposition device, an electronic device and a storage medium, wherein the real-time information superposition method comprises the following steps: acquiring video data and superposition information in a live video stream frame by frame; converting the video data to obtain a video clip; drawing each frame of canvas according to the superposition information to obtain a target canvas; and overlapping the target canvas on the corresponding video clips according to the time sequence for playing. By the method and the device, the problem that live broadcast delay is caused by the fact that a large amount of time is needed for processing generation, synthesis and processing of the superimposed information in the related technology is solved, time for synthesizing the superimposed information into the video stream is saved, delay broadcast of the live broadcast video stream in the display process is reduced, and the superimposed information is broadcast synchronously.
Description
Technical Field
The present application relates to the field of data processing technologies, and in particular, to a real-time information superimposing method and apparatus, an electronic apparatus, and a storage medium.
Background
With the rapid development of internet technology, the application of the live broadcast technology in the fields of security, traffic, interactive live broadcast and the like is wider and wider. The video live broadcast has high real-time requirements, and in the live broadcast in certain specific environments, the situation that the information is overlapped in the live broadcast stream needs to be analyzed through an algorithm exists, for example, abnormal situations can be found timely when security monitoring is needed, and online and offline interaction needs to be reflected timely in the interactive live broadcast.
The current scheme for superimposing information in a live stream is: and obtaining the superposed information after algorithm analysis, synthesizing the superposed information and the video stream into a live stream, and displaying the information to be superposed together as a part of the live stream after the browser obtains the video stream. According to different scene requirements of live broadcasting, a large amount of time is needed to process generation, synthesis and processing of the superimposed information, and delayed broadcasting of the live broadcasting can be caused.
Aiming at the problem that the generation, synthesis and processing of the superimposed information need to be processed in a large amount of time in the related technology, which can cause the delayed broadcasting of the live broadcast, no effective solution is provided at present.
Disclosure of Invention
The embodiment provides a real-time information superposition method, a real-time information superposition device, an electronic device and a storage medium, so as to solve the problem that the generation, synthesis and processing of superposition information need to be processed in a large amount of time in the related art, which can cause delayed broadcasting of live broadcast.
In a first aspect, a real-time information overlaying method is provided in this embodiment, including:
acquiring video data and superposition information in a live video stream frame by frame;
converting the video data to obtain a video clip;
drawing each frame of canvas according to the superposition information to obtain a target canvas;
and overlapping the target canvas on the corresponding video clips according to the time sequence for playing.
By the method, the problem that the generation, synthesis and processing of the superimposed information need to be processed in a large amount of time in the related technology, which can cause delayed broadcasting of live broadcast is solved, the time for synthesizing the superimposed information into the video stream is saved, delayed broadcasting of the live broadcast video stream in the display process is reduced, and the superimposed information is synchronously broadcasted.
In some embodiments, the embodiment of the present application provides the real-time information overlaying method, further including:
and acquiring a live broadcast request of a client, and acquiring a live broadcast video stream according to the live broadcast request through a live broadcast protocol.
In some embodiments, the acquiring video data and overlay information from a live video stream frame by frame includes:
and acquiring the live video stream, and performing frame-by-frame decoding and slicing on the video stream to obtain video data and superposition information of each frame.
In some embodiments, the converting the video data to obtain the video segment includes:
js is utilized to analyze the video data in JavaScript, and the analysis result is packaged into a video segment in MP4 format in real time.
In some embodiments, the drawing the canvas of each frame according to the overlay information to obtain a target canvas includes:
creating a canvas frame consistent with the width and the height of each frame of the video clip;
and drawing each frame of canvas in the canvas frame according to the superposition information to obtain a target canvas.
In some embodiments, the method for superimposing real-time information provided in the embodiments of the present application further includes, before drawing a canvas of each frame according to the superimposition information:
and screening the superposition information in response to the selection operation of the user.
In some embodiments, the embodiment of the present application provides the real-time information overlaying method, further including:
and storing the superposition information into a binary data cache region, and extracting the superposition information in the data cache region when drawing each frame of canvas.
In a second aspect, the present embodiment provides a real-time information overlaying apparatus, including an obtaining module, a converting module, a drawing module, and an overlaying playing module;
the acquisition module is used for acquiring video data and superposition information in the live video stream frame by frame;
the conversion module is used for converting the video data to obtain a video segment;
the drawing module is used for drawing each frame of canvas according to the superposition information to obtain a target canvas;
and the overlapping broadcasting module is used for overlapping the target canvas on the corresponding video clip according to the time sequence for broadcasting.
By the device, the problem that the generation, synthesis and processing of the superposition information need to be processed in a large amount of time in the related technology, which can cause delayed broadcasting of live broadcast is solved, the time for synthesizing the superposition information into the video stream is saved, delayed broadcasting of the live broadcast video stream in the display process is reduced, and the superposition information is synchronously broadcasted.
In some embodiments, the real-time information overlaying device provided in the embodiments of the present application further includes a request module;
the request module is used for acquiring a live broadcast request of a client and acquiring a live broadcast video stream according to the live broadcast request through a live broadcast protocol.
In some embodiments, the obtaining module is further configured to obtain the live video stream, and perform frame-by-frame decoding and slicing on the video stream to obtain video data and overlay information of each frame.
In some embodiments, the conversion module is further configured to parse the video data in JavaScript by flv.js, and encapsulate a parsing result into a video segment in MP4 format in real time.
In some embodiments, the rendering module is further configured to create a canvas frame that conforms to the width and height of the video segments per frame;
and drawing each frame of canvas in the canvas frame according to the superposition information to obtain a target canvas.
In some embodiments, the real-time information overlaying device provided in the embodiments of the present application further includes a screening module;
and the screening module is used for responding to the selection operation of the user to screen the superposition information.
In some embodiments, the real-time information overlaying device provided in the embodiments of the present application further includes a storage module;
the storage module is used for storing the superposition information into a binary data cache region and extracting the superposition information in the data cache region when each frame of canvas is drawn.
In a third aspect, in this embodiment, there is provided an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the processor implements the real-time information overlaying method according to the first aspect.
In a fourth aspect, in the present embodiment, there is provided a storage medium having stored thereon a computer program which, when executed by a processor, implements the real-time information overlaying method of the first aspect.
Compared with the related art, the real-time information superposition method, the real-time information superposition device, the electronic device and the storage medium provided by the embodiment acquire video data and superposition information in a live video stream frame by frame; converting the video data to obtain a video clip; drawing each frame of canvas according to the superposition information to obtain a target canvas; the target canvas is overlaid on the corresponding video segments to be played according to the time sequence, the problem that the generation, synthesis and processing of a large amount of time processing overlay information in the related technology can cause delayed playing of live broadcast is solved, the time for synthesizing the overlay information into the video stream is saved, delayed playing of the live broadcast video stream in the process of obtaining and displaying is reduced, and the overlay information is played synchronously.
The details of one or more embodiments of the application are set forth in the accompanying drawings and the description below to provide a more thorough understanding of the application.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:
fig. 1 is a block diagram of a hardware structure of a terminal device in a real-time information superposition method according to an embodiment of the present application;
fig. 2 is a flowchart of a real-time information overlaying method provided by an embodiment of the present application;
FIG. 3 is a flowchart of step S230 in FIG. 2;
fig. 4 is a block diagram of a real-time information overlaying apparatus according to an embodiment of the present application.
In the figure: 210. an acquisition module; 220. a conversion module; 230. a drawing module; 240. and a superposition broadcasting module.
Detailed Description
For a clearer understanding of the objects, aspects and advantages of the present application, reference is made to the following description and accompanying drawings.
Unless defined otherwise, technical or scientific terms used herein shall have the same general meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The use of the terms "a" and "an" and "the" and similar referents in the context of this application do not denote a limitation of quantity, either in the singular or the plural. The terms "comprises," "comprising," "has," "having," and any variations thereof, as referred to in this application, are intended to cover non-exclusive inclusions; for example, a process, method, and system, article, or apparatus that comprises a list of steps or modules (elements) is not limited to the listed steps or modules, but may include other steps or modules (elements) not listed or inherent to such process, method, article, or apparatus. Reference throughout this application to "connected," "coupled," and the like is not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. Reference to "a plurality" in this application means two or more. "and/or" describes an association relationship of associated objects, meaning that three relationships may exist, for example, "A and/or B" may mean: a exists alone, A and B exist simultaneously, and B exists alone. In general, the character "/" indicates a relationship in which the objects associated before and after are an "or". The terms "first," "second," "third," and the like in this application are used for distinguishing between similar items and not necessarily for describing a particular sequential or chronological order.
The method embodiments provided in the present embodiment may be executed in a terminal, a computer, or a similar computing device. For example, the method is executed on a terminal, and fig. 1 is a block diagram of a hardware structure of the terminal according to the real-time information superimposing method of the present embodiment. As shown in fig. 1, the terminal may include one or more processors 102 (only one shown in fig. 1) and a memory 104 for storing data, wherein the processor 102 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA. The terminal may also include a transmission device 106 for communication functions and an input-output device 108. It will be understood by those of ordinary skill in the art that the structure shown in fig. 1 is merely an illustration and is not intended to limit the structure of the terminal described above. For example, the terminal may also include more or fewer components than shown in FIG. 1, or have a different configuration than shown in FIG. 1.
The memory 104 may be used to store a computer program, for example, a software program and a module of application software, such as a computer program corresponding to the real-time information superposition method in the present embodiment, and the processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, so as to implement the method described above. The memory 104 may include high speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include memory located remotely from the processor 102, which may be connected to the terminal over a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The transmission device 106 is used to receive or transmit data via a network. The network described above includes a wireless network provided by a communication provider of the terminal. In one example, the transmission device 106 includes a Network adapter (NIC) that can be connected to other Network devices through a base station to communicate with the internet. In one example, the transmission device 106 may be a Radio Frequency (RF) module, which is used to communicate with the internet in a wireless manner.
In this embodiment, a real-time information overlaying method is provided, and fig. 2 is a flowchart of the real-time information overlaying method of this embodiment, as shown in fig. 2, the flowchart includes the following steps:
step S210, acquiring video data and superposition information in a live video stream frame by frame;
step S220, converting the video data to obtain a video segment;
step S230, drawing each frame of canvas according to the superposition information to obtain a target canvas;
and step S240, overlapping the target canvas on the corresponding video clip for playing according to the time sequence.
It should be noted that the present application may be implemented on the basis of a web side, and superimposes information frame by frame. For example, the superimposition information may be superimposed every frame, and the above-described steps are performed once every 30 milliseconds, that is, every 30 milliseconds. In other embodiments, the overlay information may be overlaid every threshold time frame, such as 3 frames, and the video data and the overlay information in the live video stream are acquired every 3 frames. The shorter the time interval, the more accurate the display of the superimposed information in the finally played video.
Both the video data and the overlay information are taken from the live video stream. The overlay information refers to information which needs to be drawn on a canvas and finally overlaid on the video clip for display in broadcasting. It may be any information obtained by analyzing the live video stream, specifically, the lane area of the road surface, the detected vehicle information (color, brand, vehicle type, vehicle speed, vehicle coordinates, etc.), the lane line, etc. The video data refers to relevant video data for playing, such as image data, audio data, and the like. In this embodiment, the video data is converted to obtain a video clip supporting the player to play; the scheme of conversion may be different in order to be suitable for different players. For example, to obtain a video clip that supports direct video playback, the conversion process is: js is utilized to analyze the video data in JavaScript, and the analysis result is packaged into a video segment in MP4 format in real time. Then in the client, Media Source Extensions API parsing of html5 is called and passed to the video tag for direct playback. In other embodiments, the scheme for the conversion is not limited.
Each frame of video data or video clip corresponds to a drawing of a canvas. The target canvas is drawn according to the overlay information. Such as: the superimposed information is vehicle information (color, brand, vehicle type, vehicle speed, vehicle coordinates, etc.); then the color, brand, vehicle type, vehicle speed, etc. are drawn at the corresponding location on the canvas according to the vehicle coordinates. And then the target canvas is overlaid on the corresponding video clip for playing according to the time sequence. Specifically, the target canvas of each frame may be spliced in time sequence, the video clips of each frame may be spliced in time sequence, and when playing, the target canvas and the video clips of each frame may be aligned in time, and the target canvas may be overlaid on the corresponding video clips to be played. In this embodiment, the target canvas and the video clip are two relatively independent parts, and the target canvas is only overlapped on the corresponding video clip to present the broadcast during the broadcast, so that the time for synthesizing the overlapped information into the video stream is saved, the delayed broadcast of the live video stream in the display process is reduced, and the overlapped information is synchronously broadcast.
Through the steps, the problem that the generation, synthesis and processing of the superimposed information need to be processed in a large amount of time in the related technology, which can cause delayed broadcasting of live broadcast is solved, the time for synthesizing the superimposed information into the video stream is saved, delayed broadcasting of the live broadcast video stream in the display process is reduced, and the superimposed information is synchronously broadcasted.
In some embodiments, the real-time information overlaying method provided in the embodiments of the present application further includes the following steps:
and acquiring a live broadcast request of a client, and acquiring a live broadcast video stream according to the live broadcast request through a live broadcast protocol.
Specifically, the live broadcast protocol can be an HTTP-FLV protocol or a WebSocket-FLV protocol; and acquiring the live broadcast video stream according to the live broadcast request through an HTTP-FLV protocol or a WebSocket-FLV protocol. The HTTP-FLV encapsulates RTMP on the HTTP protocol, so that firewall penetration can be better realized. The transmission data is the tag of the FLV file. The HTTP-FLV is a file of an infinite HTTP stream. The WebSocket-FLV transmits the FLV based on the WebSocket and depends on a browser to support playing of the FLV. WebSocket is established on top of HTTP, and HTTP connection is established before WebSocket connection is established.
In some embodiments, step S210 includes the following steps:
and acquiring the live video stream, and performing frame-by-frame decoding and slicing on the video stream to obtain video data and superposition information of each frame.
Specifically, the live video stream is acquired, and frame-by-frame decoding and fragmentation are performed on the video stream by using flv.js, so that video data and superimposed information of each frame can be quickly obtained, and delayed broadcasting of the live video stream in the process of acquiring and displaying is reduced.
In some embodiments, as shown in fig. 3, step S230 includes the following steps:
step S231, creating a canvas frame with the same width and height as those of each frame of the video clip;
and step S232, drawing each frame of canvas in the canvas frame according to the superposition information to obtain a target canvas.
Specifically, the width and height of each frame of video clips are obtained, and a canvas frame is created according to the height and the width so as to create the canvas frame consistent with the width and the height of each frame of video clips. And in the canvas frame, drawing the canvas of each frame according to the superposition information. In this embodiment, the Canvas represents an HTML Canvas element- < Canvas >, which has a getContext method, and by using this method, an environment object for drawing on the Canvas can be created, and the object has various drawing paths, rectangles, circles, characters and methods for adding images, so that drawing lines, curves, fills, edges, and the like can be performed according to the superimposed information to obtain the target Canvas. Therefore, the drawing of the target Canvas is quickly completed within the time of each frame, and the Canvas method is used to ensure that the Canvas still has better performance under the condition that more information needs to be displayed. The final superposition effect is completed by Canvas (Canvas), and the performance is better at the browser end. The rich drawing method of the Canvas (Canvas) enables the overlay information to be displayed in a variety of ways, and is convenient for subsequent modification.
In some embodiments, in order to improve the display expandability, diversity and interchangeability, before drawing the canvas of each frame according to the overlay information, the real-time information overlay method provided in the embodiments of the present application further includes the following steps:
and screening the superposition information in response to the selection operation of the user.
Specifically, the selection operation may be a specific operation such as clicking, sliding, and moving. For example, a user interaction button such as a first option, a second option, a third option and the like is arranged on the playing interface, and the first option is used for shielding the vehicle attribute; a second option is for an area frame that shields the roadway; a third option for masking the bounding box of the on-road vehicle; then based on the specific operation of the even number, the shielding of the relevant information can be completed by selecting the corresponding option, so that the superposed information is screened out. Such as: and clicking the second option, deleting the area frame of the road surface from the superposed information, thereby finishing the screening of the superposed information and controlling the content displayed on the canvas. For another example, if the third option is slid to the second option, the area frame of the road surface and the surrounding frame of the vehicle are deleted from the superimposed information, so that the screening of the superimposed information is completed, and the content displayed on the canvas can be controlled. For another example, the third option is moved to the frame, so that the frame surrounding the vehicle is deleted from the superimposed information, thereby quickly completing the screening of the superimposed information, controlling the content displayed on the canvas, and facilitating the selection operation of the user on the screening of the superimposed information. When the third option is moved to the frame, a prompt message can be displayed, the prompt message is 'delete the surrounding frame of the vehicle', so that the user can quickly know the screened object, and the user can use the vehicle conveniently. For the selection operation of screening the superimposed information, no one example is given here.
The playing interface provides some user interaction buttons, so that a user can conveniently switch information to be displayed according to requirements, and the superimposed information changes frame by frame along with the picture of the video, so that the user can quickly respond to the interaction operation of the user.
In some embodiments, the real-time information overlaying method provided in the embodiments of the present application further includes:
and storing the superposition information into a binary data cache region, and extracting the superposition information in the data cache region when drawing each frame of canvas.
Specifically, the data buffer may be an ArrayBuffer object, which is a container for storing binary information by the client, and the client may call a method carried by the object to read the superimposed information therein. Such as: the client can decompose the binary information byte by byte through the DataView view, so that the superposition information is obtained. The character string can be converted through a mapping table, so that the superposed information is obtained; for example, a piece of superimposed information corresponds to the color of a car, the client analyzes the information to obtain 1, and the superimposed information is obtained by using 0 for white, 1 for black and 2 for red, which means that the car is black. If the vehicle needs to be drawn on the canvas, the position of the vehicle on the canvas needs to be known, and the position information is the coordinates of the target on the canvas. In other embodiments, the overlay information may be stored in a database, and the relevant overlay information may be extracted when in use.
It should be noted that the steps illustrated in the above-described flow diagrams or in the flow diagrams of the figures may be performed in a computer system, such as a set of computer-executable instructions, and that, although a logical order is illustrated in the flow diagrams, in some cases, the steps illustrated or described may be performed in an order different than here.
In this embodiment, a real-time information overlaying device is further provided, and the device is used to implement the foregoing embodiments and preferred embodiments, which have already been described and will not be described again. The terms "module," "unit," "subunit," and the like as used below may implement a combination of software and/or hardware for a predetermined function. Although the means described in the embodiments below are preferably implemented in software, an implementation in hardware, or a combination of software and hardware is also possible and contemplated.
Fig. 4 is a block diagram of the real-time information superimposing apparatus of the present embodiment, and as shown in fig. 4, the apparatus includes: an acquisition module 210, a conversion module 220, a drawing module 230 and a superposition broadcasting module 240;
the acquiring module 210 is configured to acquire video data and overlay information in a live video stream frame by frame;
the conversion module 220 is configured to convert the video data to obtain a video segment;
the drawing module 230 is configured to draw each frame of canvas according to the overlay information to obtain a target canvas;
and the overlaying playing module 240 is configured to overlay the target canvas on the corresponding video segment according to a time sequence for playing.
By the device, the problem that the generation, synthesis and processing of the superposition information need to be processed in a large amount of time in the related technology, which can cause delayed broadcasting of live broadcast is solved, the time for synthesizing the superposition information into the video stream is saved, delayed broadcasting of the live broadcast video stream in the display process is reduced, and the superposition information is synchronously broadcasted.
In some embodiments, the obtaining module 210 is further configured to obtain the live video stream, perform frame-by-frame decoding and slicing on the video stream, and obtain video data and overlay information of each frame.
In some embodiments, the converting module 220 is further configured to parse the video data in JavaScript by flv.
In some embodiments, the rendering module 230 is further configured to create a canvas frame that conforms to the width and height of each frame of the video segment;
and drawing each frame of canvas in the canvas frame according to the superposition information to obtain a target canvas.
In some embodiments, on the basis of fig. 4, the real-time information overlaying device provided in the embodiments of the present application further includes a request module;
and the request module is used for acquiring a live broadcast request of the client and acquiring a live broadcast video stream according to the live broadcast request through a live broadcast protocol.
In some embodiments, on the basis of fig. 4, the real-time information overlaying device provided in the embodiments of the present application further includes a screening module;
and the screening module is used for responding to the selection operation of the user to screen the superposed information before drawing the canvas of each frame according to the superposed information.
In some embodiments, on the basis of fig. 4, the real-time information overlaying device provided in the embodiments of the present application further includes a storage module;
and the storage module is used for storing the superposition information into a binary data cache region and extracting the superposition information in the data cache region when each frame of canvas is drawn.
The above modules may be functional modules or program modules, and may be implemented by software or hardware. For a module implemented by hardware, the modules may be located in the same processor; or the modules can be respectively positioned in different processors in any combination.
There is also provided in this embodiment an electronic device comprising a memory having a computer program stored therein and a processor arranged to run the computer program to perform the steps of any of the above method embodiments.
Optionally, the electronic apparatus may further include a transmission device and an input/output device, wherein the transmission device is connected to the processor, and the input/output device is connected to the processor.
Optionally, in this embodiment, the processor may be configured to execute the following steps by a computer program:
s1, acquiring video data and superposition information in the live video stream frame by frame;
s2, converting the video data to obtain video segments;
s3, drawing each frame of canvas according to the superposition information to obtain a target canvas;
and S4, overlapping the target canvas on the corresponding video clips in time sequence for playing.
Through the electronic device, the problem that the generation, synthesis and processing of the superimposed information need to be processed in a large amount of time in the related technology, which can cause delayed broadcasting of live broadcast is solved, the time for synthesizing the superimposed information into the video stream is saved, delayed broadcasting of the live broadcast video stream in the process of obtaining and displaying is reduced, and the superimposed information is broadcasted synchronously.
It should be noted that, for specific examples in this embodiment, reference may be made to the examples described in the foregoing embodiments and optional implementations, and details are not described again in this embodiment.
In addition, in combination with the real-time information superimposing method provided in the foregoing embodiment, a storage medium may also be provided in this embodiment. The storage medium having stored thereon a computer program; the computer program, when executed by a processor, implements any of the real-time information overlaying methods of the above embodiments.
It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to be limiting. All other embodiments, which can be derived by a person skilled in the art from the examples provided herein without any inventive step, shall fall within the scope of protection of the present application.
It is obvious that the drawings are only examples or embodiments of the present application, and it is obvious to those skilled in the art that the present application can be applied to other similar cases according to the drawings without creative efforts. Moreover, it should be appreciated that in the development of any such actual implementation, as in any engineering or design project, numerous implementation-specific decisions must be made to achieve the developers' specific goals, such as compliance with system-related and business-related constraints, which may vary from one implementation to another.
The term "embodiment" is used herein to mean that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the present application. The appearances of such phrases in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is to be expressly or implicitly understood by one of ordinary skill in the art that the embodiments described in this application may be combined with other embodiments without conflict.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the patent protection. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.
Claims (10)
1. A method for superimposing real-time information, comprising:
acquiring video data and superposition information in a live video stream frame by frame;
converting the video data to obtain a video clip;
drawing each frame of canvas according to the superposition information to obtain a target canvas;
and overlapping the target canvas on the corresponding video clips according to the time sequence for playing.
2. The real-time information overlaying method of claim 1, further comprising:
and acquiring a live broadcast request of a client, and acquiring a live broadcast video stream according to the live broadcast request through a live broadcast protocol.
3. The method according to claim 2, wherein the acquiring video data and overlay information in the live video stream frame by frame comprises:
and acquiring the live video stream, and performing frame-by-frame decoding and slicing on the video stream to obtain video data and superposition information of each frame.
4. The method of claim 1, wherein the converting the video data to obtain a video segment comprises:
js is utilized to analyze the video data in JavaScript, and the analysis result is packaged into a video segment in MP4 format in real time.
5. The real-time information superposition method according to claim 1, wherein the step of drawing each frame of canvas according to the superposition information to obtain a target canvas comprises the steps of:
creating a canvas frame consistent with the width and the height of each frame of the video clip;
and drawing each frame of canvas in the canvas frame according to the superposition information to obtain a target canvas.
6. The method of claim 1, further comprising, before rendering the canvas of each frame according to the overlay information:
and screening the superposition information in response to the selection operation of the user.
7. The real-time information overlaying method of claim 1, further comprising:
and storing the superposition information into a binary data cache region, and extracting the superposition information in the data cache region when drawing each frame of canvas.
8. A real-time information superposition device is characterized by comprising an acquisition module, a conversion module, a drawing module and a superposition broadcasting module;
the acquisition module is used for acquiring video data and superposition information in the live video stream frame by frame;
the conversion module is used for converting the video data to obtain a video segment;
the drawing module is used for drawing each frame of canvas according to the superposition information to obtain a target canvas;
and the overlapping broadcasting module is used for overlapping the target canvas on the corresponding video clip according to the time sequence for broadcasting.
9. An electronic device comprising a memory and a processor, wherein the memory has stored therein a computer program, and the processor is configured to execute the computer program to perform the real-time information overlaying method of any of claims 1 to 7.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the real-time information overlaying method according to any one of the claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111143264.9A CN114071228A (en) | 2021-09-28 | 2021-09-28 | Real-time information superposition method and device, electronic device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111143264.9A CN114071228A (en) | 2021-09-28 | 2021-09-28 | Real-time information superposition method and device, electronic device and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114071228A true CN114071228A (en) | 2022-02-18 |
Family
ID=80233801
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111143264.9A Pending CN114071228A (en) | 2021-09-28 | 2021-09-28 | Real-time information superposition method and device, electronic device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114071228A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115767031A (en) * | 2022-11-17 | 2023-03-07 | 北京众谊越泰科技有限公司 | Low-delay video monitoring image display method and system for moving loop system |
| CN115914748A (en) * | 2022-10-18 | 2023-04-04 | 阿里云计算有限公司 | Visual display method and device for visual recognition result and electronic equipment |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103702040A (en) * | 2013-12-31 | 2014-04-02 | 广州华多网络科技有限公司 | Real-time video graphic decoration superposing processing method and system |
| CN107888845A (en) * | 2017-11-14 | 2018-04-06 | 腾讯数码（天津）有限公司 | A kind of method of video image processing, device and terminal |
| CN110738846A (en) * | 2019-09-27 | 2020-01-31 | 同济大学 | Vehicle behavior monitoring system based on radar and video group and implementation method thereof |
| CN111193937A (en) * | 2020-01-15 | 2020-05-22 | 北京拙河科技有限公司 | Processing method, device, equipment and medium for live video data |
| CN112929684A (en) * | 2021-01-27 | 2021-06-08 | 广州虎牙科技有限公司 | Video overlay information updating method and device, electronic equipment and storage medium |
-
2021
- 2021-09-28 CN CN202111143264.9A patent/CN114071228A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103702040A (en) * | 2013-12-31 | 2014-04-02 | 广州华多网络科技有限公司 | Real-time video graphic decoration superposing processing method and system |
| CN107888845A (en) * | 2017-11-14 | 2018-04-06 | 腾讯数码（天津）有限公司 | A kind of method of video image processing, device and terminal |
| CN110738846A (en) * | 2019-09-27 | 2020-01-31 | 同济大学 | Vehicle behavior monitoring system based on radar and video group and implementation method thereof |
| CN111193937A (en) * | 2020-01-15 | 2020-05-22 | 北京拙河科技有限公司 | Processing method, device, equipment and medium for live video data |
| CN112929684A (en) * | 2021-01-27 | 2021-06-08 | 广州虎牙科技有限公司 | Video overlay information updating method and device, electronic equipment and storage medium |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115914748A (en) * | 2022-10-18 | 2023-04-04 | 阿里云计算有限公司 | Visual display method and device for visual recognition result and electronic equipment |
| CN115767031A (en) * | 2022-11-17 | 2023-03-07 | 北京众谊越泰科技有限公司 | Low-delay video monitoring image display method and system for moving loop system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108737882B (en) | Image display method, image display device, storage medium and electronic device | |
| CN110647303B (en) | Multimedia playing method and device, storage medium and electronic equipment | |
| CN101656755B (en) | Method and device for transmitting and broadcasting advertising programs | |
| CN102577366B (en) | For distributing the system and method for the auxiliary data be embedded in video data | |
| CN114071228A (en) | Real-time information superposition method and device, electronic device and storage medium | |
| CN108632676B (en) | Image display method, image display device, storage medium and electronic device | |
| CN105163164B (en) | Information shows item methods of exhibiting and device | |
| CN108200447B (en) | Live broadcast data transmission method and device, electronic equipment, server and storage medium | |
| CN105142000B (en) | Information-pushing method and system based on content of televising | |
| CN110784730B (en) | Live video data transmission method, device, equipment and storage medium | |
| KR101850482B1 (en) | System and method for providing providing augmented reality service associated with broadcasting | |
| CN103957324A (en) | Method and system for interaction with television programs through mobile communication terminal | |
| JP2001527724A (en) | Method of embedding links to networked resources in transmission media | |
| CN105812957A (en) | Continuous playing method and device for multimedia file | |
| CN103533394A (en) | Information pushing method and device and server | |
| CN112929680A (en) | Live broadcast room image rendering method and device, computer equipment and storage medium | |
| CN110267093B (en) | Live video pushing method and device, storage medium, terminal and live mirror | |
| CN112689168A (en) | Dynamic effect processing method, dynamic effect display method and dynamic effect processing device | |
| CN104185040A (en) | Application synchronization method, application server and terminal | |
| US20050066366A1 (en) | Receiving apparatus, method for displaying in conjunction with television program, and printing control method | |
| CN111050204A (en) | Video clipping method and device, electronic equipment and storage medium | |
| CN106412617B (en) | Remote debugging control method and device | |
| KR101915792B1 (en) | System and Method for Inserting an Advertisement Using Face Recognition | |
| CN102137253A (en) | Picture processing method, terminal and server | |
| CN107529075B (en) | Cross-screen continuous playing method, system, server, equipment and cross-screen continuous playing equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20220218 |