CN106375793A - Superposition method and superposition system of video structured information, and user terminal - Google Patents
Superposition method and superposition system of video structured information, and user terminal Download PDFInfo
- Publication number
- CN106375793A CN106375793A CN201610763495.2A CN201610763495A CN106375793A CN 106375793 A CN106375793 A CN 106375793A CN 201610763495 A CN201610763495 A CN 201610763495A CN 106375793 A CN106375793 A CN 106375793A
- Authority
- CN
- China
- Prior art keywords
- video
- frame
- structured message
- user terminal
- timestamp
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 92
- 230000008569 process Effects 0.000 claims abstract description 50
- 238000012544 monitoring process Methods 0.000 claims abstract description 37
- 238000012545 processing Methods 0.000 claims description 71
- 238000009877 rendering Methods 0.000 claims description 22
- 238000012163 sequencing technique Methods 0.000 claims description 19
- 230000008859 change Effects 0.000 claims description 12
- 239000000284 extract Substances 0.000 claims description 9
- 238000004891 communication Methods 0.000 claims description 7
- 238000010030 laminating Methods 0.000 claims description 6
- 238000000605 extraction Methods 0.000 claims description 3
- 239000000654 additive Substances 0.000 description 15
- 230000000996 additive effect Effects 0.000 description 15
- 230000001360 synchronised effect Effects 0.000 description 8
- 238000012546 transfer Methods 0.000 description 5
- 230000000052 comparative effect Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 230000008878 coupling Effects 0.000 description 2
- 238000000151 deposition Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000003860 storage Methods 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000003139 buffering effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
- H04N21/254—Management at additional data server, e.g. shopping server, rights management server
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/435—Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
- H04N21/4351—Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream involving reassembling additional data, e.g. rebuilding an executable program from recovered modules
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
- H04N5/265—Mixing
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Databases & Information Systems (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The invention provides a superposition method and a superposition system of video structured information, and a user terminal. The method comprises the following steps: the user terminal receives at least one video frame, wherein the video frame is synchronously sent by a video source server to the user terminal and an algorithm server; meanwhile, the user terminal further receives structured information sent by the algorithm server, wherein the structured information is generated by the algorithm server by identification of a monitoring target in the video frame and in accordance with the monitoring target; and finally, the user terminal superposes custom information in the video frame according to a timestamp of the video frame, a video timestamp in the structured information and a position coordinate of the custom information. In the superposition method provided by the embodiment of the invention, the algorithm server is only used for generating the structured information without superposing the structured information, the superposition process of the structured information is executed by the user terminal, so that the superposition speed of the structured information is accelerated, thereby both guaranteeing the display instantaneity of video stream and reducing the requirements of the superposition system for the software and hardware configuration.
Description
Technical field
The present invention relates to technical field of video processing, in particular to a kind of stacking method of video structural information,
User terminal and overlapping system.
Background technology
At present, developing rapidly with video processing technique, can meet different user to regarding by video superimpose technology
The different demands of the self-defined information in frequency, some self-defined informations are added to the position specified in particular video frequency frame in video
Put, then the video being superimposed with the information of making by oneself is presented to user, e.g., a certain subject information is subscribed in user terminal application, now needs
This subject information is added in video, then the video being superimposed with this subject information is sent to corresponding user terminal.
Currently, provide a kind of stacking method of video structural information in correlation technique, the method is mainly: first,
Video source server by video stream to arithmetic server, then, directly folded by arithmetic server by the structured message of output
It is added in video on corresponding image, and the video distribution of structured message will be superimposed with to streaming media service through secondary coding
In device, finally, the video flowing that user terminal directly pulls in this streaming media server plays out.Need to increase a stream matchmaker
Body server, thus, increased system hardware and software deployment cost；In addition, when arithmetic server carries out the process of real-time video, no
Only need to generate structured message to be superimposed in addition it is also necessary to be superimposed to this structured message to be superimposed in video, need big
The efficiency calculating and processing measured needs identical with the frame per second of video, the common computer of the speed that at least will ensure process in real time
It is difficult to accomplish, also needs to meet multi-channel video simultaneously and carry out simultaneously, thus requirement must be special server to complete this parallel
A little tasks.
During realizing the present invention, inventor finds in correlation technique at least there is problems in that in correlation technique
Video information superimposing technique need to generate the structured message related to video flowing by arithmetic server, simultaneously by this structuring
Information superposition to the corresponding video frame of video flowing specific location, during real time video processing, at arithmetic server
Reason data volume is big and needs to ensure the real-time that video shows, it is then desired to ensure video information stack velocity, and then to video
The configuration requirement of information superposition system is high.
Content of the invention
In view of this, the purpose of the embodiment of the present invention is to provide a kind of stacking method of video structural information, user
Terminal and overlapping system, to improve the stack velocity of structured message, thus realizing both having ensured the real-time that video flowing shows, and
Reduce the requirement to overlapping system software and hardware configuration.
In a first aspect, embodiments providing a kind of stacking method of video structural information, the method includes:
User terminal receives at least one frame of video, and described frame of video is sent to described user for video source server sync
Terminal and arithmetic server；
Described user terminal receives the structured message that described arithmetic server sends, and described structured message is described calculation
Method server identify described frame of video in monitoring objective and according to described monitoring objective generate, described structured message includes
The position coordinateses of video time stamp, the corresponding self-defined information of described monitoring objective and described self-defined information；
Described user terminal will be described according to the timestamp of described frame of video, described video time stamp and described position coordinateses
Self-defined information is superimposed in described frame of video.
In conjunction with a first aspect, embodiments providing the first possible embodiment of first aspect, wherein, institute
State user terminal and respectively described frame of video and described structured message are received by different threads.
In conjunction with a first aspect, the possible embodiment of the second that embodiments provides first aspect, wherein, institute
State after user terminal receives at least one frame of video, also include: caching process is carried out to frame of video each described, and right one by one
Described frame of video after caching is decoded processing with surface rendering, according to the sequencing output surface wash with watercolours frame by frame of surface rendering
Described frame of video after dye；
After described user terminal receives the structured message that described arithmetic server sends, also include: by receive
Multiple described structured messages are sequentially stored in buffer queue.
In conjunction with the possible embodiment of the second of first aspect, embodiments provide first aspect the third
Possible embodiment, wherein, described user terminal is according to the timestamp of described frame of video, described video time stamp and institute's rheme
Put coordinate described self-defined information is superimposed in described frame of video, comprising:
Timestamp according to described frame of video and described video time stamp are chosen and described video from described buffer queue
The corresponding described structured message of frame；
Described self-defined information and described position coordinateses are extracted from described structured message；
Described self-defined information is superimposed at the described position coordinateses in described frame of video.
In conjunction with the third possible embodiment of first aspect, embodiments provide the 4th kind of first aspect
Possible embodiment, wherein, the described timestamp according to described frame of video and described video time stamp are from described buffer queue
Before middle selection described structured message corresponding with described frame of video, also include:
Judge to whether there is described structured message in described buffer queue, if existing, according to being stored in described caching team
The sequencing of row chooses a described structured message one by one, extracts described video time stamp from described structured message.
In conjunction with the 4th kind of possible embodiment of first aspect, embodiments provide the 5th kind of first aspect
Possible embodiment, wherein, the described timestamp according to described frame of video and described video time stamp are from described buffer queue
Middle selection described structured message corresponding with described frame of video, including:
Timestamp according to described video time stamp and described frame of video determines the processing mode to described structured message；
Using described processing mode, described structured message is processed accordingly, described processing mode includes: by institute
State structured message and return described buffer queue as structured message corresponding with described frame of video, by described structured message
In or described structured message removed from described buffer queue；
When determining that described processing mode is to believe described structured message as structuring corresponding with described frame of video
Cease or determine that described processing mode is by when in the described structured message described buffer queue of return, terminate choosing next
The operation of individual described structured message, and choose corresponding structured message for a later frame of described frame of video.
In conjunction with the 5th kind of possible embodiment of first aspect, embodiments provide the 6th kind of first aspect
Possible embodiment, wherein, the described timestamp according to described video time stamp and described frame of video determines to described structure
The processing mode of change information, including:
When described video time stamp is consistent with the timestamp of described frame of video, determine the process to described structured message
Mode is as structured message corresponding with described frame of video using described structured message；
When the timestamp of described video time stamp and described frame of video is inconsistent, judge whether described video time stamp is big
In the timestamp of described frame of video, if it is determined that the processing mode to described structured message is by described structured message
Return in described buffer queue, if not it is determined that to the processing mode of described structured message be by described structured message from
Remove in described buffer queue.
Second aspect, the embodiment of the present invention additionally provides a kind of user terminal, and this user terminal includes:
Frame of video receiver module, for receiving at least one frame of video, described frame of video is sent out for video source server sync
Give described user terminal and arithmetic server；
Structured message receiver module, for receiving the structured message that described arithmetic server sends, described structuring
Information is monitoring objective that described arithmetic server identifies in described frame of video and is generated according to described monitoring objective, described knot
Structure information includes the position of video time stamp, the corresponding self-defined information of described monitoring objective and described self-defined information
Coordinate；
Information superposition module, for according to the timestamp of described frame of video, described video time stamp and described position coordinateses
Described self-defined information is superimposed in described frame of video.
In conjunction with second aspect, embodiments provide the first possible embodiment of second aspect, wherein, institute
State user terminal and respectively described frame of video and described structured message are received by different threads.
In conjunction with second aspect, embodiments provide the possible embodiment of the second of second aspect, wherein, institute
State user terminal also to include:
Frame of video processing module, for caching process is carried out to frame of video each described, and one by one to caching after described in
Frame of video is decoded and surface rendering is processed, according to surface rendering sequencing after output surface renders frame by frame described in regard
Frequency frame；
Structured message cache module, for being sequentially stored into buffer queue by the multiple described structured message receiving
In.
In conjunction with the possible embodiment of the second of second aspect, embodiments provide second aspect the third
Possible embodiment, wherein, described information laminating module includes:
Choose unit, select from described buffer queue for the timestamp according to described frame of video and described video time stamp
Take described structured message corresponding with described frame of video；
Extraction unit, for extracting described self-defined information and described position coordinateses from described structured message；
Superpositing unit, at the described position coordinateses that are superimposed to described self-defined information in described frame of video.
In conjunction with the third possible embodiment of second aspect, embodiments provide the 4th kind of second aspect
Possible embodiment, wherein, described information laminating module also includes:
Judging unit, for judging to whether there is described structured message in described buffer queue, if exist, according to depositing
The sequencing entering described buffer queue chooses a described structured message one by one, extracts described from described structured message
Video time stamp.
In conjunction with the 4th kind of possible embodiment of second aspect, embodiments provide the 5th kind of second aspect
Possible embodiment, wherein, described selection unit includes:
Processing mode determination subelement, determines to institute for the timestamp according to described video time stamp and described frame of video
State the processing mode of structured message；
Structured message chooses subelement, for being located accordingly to described structured message using described processing mode
Reason, described processing mode includes: using described structured message as structured message corresponding with described frame of video, by described knot
Structure information returns in described buffer queue or removes described structured message from described buffer queue；When determining
Described processing mode is as structured message corresponding with described frame of video or to determine described using described structured message
When processing mode is to return described structured message in described buffer queue, terminate choosing next described structured message
Operation, and choose corresponding structured message for a later frame of described frame of video.
In conjunction with the 5th kind of possible embodiment of second aspect, embodiments provide the 6th kind of second aspect
Possible embodiment, wherein, described processing mode determination subelement, specifically for when described video time stamp and described video
When the timestamp of frame is consistent, determine that the processing mode to described structured message is to regard described structured message as with described
The corresponding structured message of frequency frame；When the timestamp of described video time stamp and described frame of video is inconsistent, regard described in judgement
Frequency timestamp whether more than the timestamp of described frame of video, if it is determined that to the processing mode of described structured message be by
Described structured message returns in described buffer queue, if not it is determined that the processing mode to described structured message is by institute
State structured message to remove from described buffer queue.
The third aspect, the embodiment of the present invention additionally provides a kind of overlapping system of video structural information, and this system includes:
Video source server, arithmetic server and user terminal, described video source server respectively with described arithmetic server and described
User terminal communication connects, and described user terminal is communicated to connect with described arithmetic server；
Described video source server, for being sent to described user terminal and described algorithm by least one video frame synchronization
Server；
Described arithmetic server, for receiving described frame of video, and identifies the monitoring objective in described frame of video, according to institute
State monitoring objective and generate the corresponding structured message of described frame of video, described structured message is transmitted to described user terminal,
Described structured message includes video time stamp, the corresponding self-defined information of described monitoring objective and described self-defined information
Position coordinateses；
Described user terminal is used for receiving the structured message that described frame of video and described arithmetic server send, and according to
Described self-defined information is superimposed to described video by the timestamp of described frame of video, described video time stamp and described position coordinateses
In frame.
In the stacking method of video structural information provided in an embodiment of the present invention, user terminal and overlapping system, should
Method includes: user terminal receives at least one frame of video, and this frame of video is sent to user terminal for video source server sync
With arithmetic server；Meanwhile, the structured message that this user terminal also receiving algorithm server sends, this structured message is
Arithmetic server identify frame of video in monitoring objective and according to monitoring objective generate；Finally, user terminal is according to frame of video
The position coordinateses of timestamp, the video time stamp in structured message and self-defined information self-defined information is superimposed to video
In frame.In embodiments of the present invention, first, video source server is synchronous to arithmetic server and user terminal with frame of video form
Transmission live video stream, arithmetic server is only used for generating the corresponding structured message of each frame of video, and then, user terminal is same
When receive video flowing and the corresponding structured message of each frame of video, and each structured message is superimposed to corresponding frame of video
In, during whole video structural information superposition, by the superposition of the generating process of structured message and structured message
Process is separated, and arithmetic server need not carry out structured message superposition, and structured message additive process is come by user terminal
Execution, and structured message additive process is shared on multiple user terminals, the superposition of different user terminal structured message
Data processing amount also greatly reduces, and improves the stack velocity of structured message, thus it is real-time both to have ensure that video flowing showed
Property, reduce the requirement to overlapping system software and hardware configuration again.
For enabling the above objects, features and advantages of the present invention to become apparent, preferred embodiment cited below particularly, and coordinate
Appended accompanying drawing, is described in detail below.
Brief description
In order to be illustrated more clearly that the technical scheme of the embodiment of the present invention, below will be attached to use required in embodiment
Figure is briefly described it will be appreciated that the following drawings illustrate only certain embodiments of the present invention, and it is right to be therefore not construed as
The restriction of scope, for those of ordinary skill in the art, on the premise of not paying creative work, can also be according to this
A little accompanying drawings obtain other related accompanying drawings.
Fig. 1 shows that a kind of flow process of the stacking method of video structural information that the embodiment of the present invention is provided is illustrated
Figure；
Fig. 2 shows that the flow process of the stacking method of preferred video structural information that the embodiment of the present invention is provided is illustrated
Figure；
Fig. 3 shows a kind of structural representation of user terminal that the embodiment of the present invention is provided；
Fig. 4 shows a kind of structural representation of the overlapping system of video structural information that the embodiment of the present invention is provided
Figure；
Fig. 5 shows the structural representation of the overlapping system of another kind of video structural information that the embodiment of the present invention is provided
Figure.
Specific embodiment
Purpose, technical scheme and advantage for making the embodiment of the present invention are clearer, below in conjunction with the embodiment of the present invention
Middle accompanying drawing, the technical scheme in the embodiment of the present invention is clearly and completely described it is clear that described embodiment only
It is a part of embodiment of the present invention, rather than whole embodiments.The present invention generally described and illustrated in accompanying drawing herein is real
The assembly applying example can be arranged with various different configurations and design.Therefore, below to the present invention's providing in the accompanying drawings
The detailed description of embodiment is not intended to limit the scope of claimed invention, but is merely representative of the selected reality of the present invention
Apply example.Based on embodiments of the invention, the institute that those skilled in the art are obtained on the premise of not making creative work
There is other embodiment, broadly fall into the scope of protection of the invention.
Need related to video flowing by arithmetic server generation in view of the video information superimposing technique in correlation technique
This structured message is superimposed to the specific location of the corresponding video frame of video flowing, due to regarding in real time by structured message simultaneously
In frequency processing procedure, arithmetic server processing data amount is big and needs to ensure the real-time that video shows, it is then desired to ensure to regard
Frequency information superposition speed, so high to the configuration requirement of video information overlapping system.Based on this, embodiments provide one
Plant stacking method, user terminal and the overlapping system of video structural information, be described below by embodiment.
As shown in figure 1, embodiments providing a kind of stacking method of video structural information, the method includes walking
Rapid s102-s106, specific as follows:
Step s102: user terminal receives at least one frame of video, this frame of video is sent to for video source server sync
Above-mentioned user terminal and arithmetic server；
Step s104: above-mentioned user terminal receives the structured message that above-mentioned arithmetic server sends, this structured message
For the monitoring objective in the above-mentioned arithmetic server above-mentioned frame of video of identification and according to the generation of above-mentioned monitoring objective, this structuring is believed
Breath includes the position coordinateses of video time stamp, the corresponding self-defined information of monitoring objective and above-mentioned self-defined information；
Step s106: above-mentioned user terminal is according to the timestamp of above-mentioned frame of video, above-mentioned video time stamp and above-mentioned position
Above-mentioned self-defined information is superimposed in above-mentioned frame of video coordinate.
Wherein, the detailed process that above-mentioned arithmetic server generates the corresponding structured message of each frame of video is: algorithm clothes
After business device receives the live video stream that video source server sends, one by one each frame of video is decoded, regards decoded
Frequency frame (yuv data) is transferred to algorithm, and algorithm detects and identifies and whether there is monitoring mesh in each decoded video frame images
Mark, if it is present output algorithm result, the i.e. corresponding structured message of this frame of video, wherein, monitoring objective is pre-defined
Some target types, this monitoring objective can be face, vehicle etc., and above-mentioned video time stamp is the timestamp of this frame of video,
The corresponding self-defined information of above-mentioned monitoring objective can be set in advance according to the type according to monitoring objective, this self-defined letter
Breath includes: to the textual description of target, the positional information of target etc., the position coordinateses of above-mentioned self-defined information can be monitoring
Position coordinateses in video frame images for the target or the type position coordinateses set in advance according to monitoring objective.
In the embodiment that the present invention provides, first, video source server is with frame of video form to arithmetic server and use
Family terminal synchronous transfer live video stream, arithmetic server is only used for generating the corresponding structured message of each frame of video, then,
User terminal receives video flowing and the corresponding structured message of each frame of video simultaneously, and each structured message is superimposed to phase
In the frame of video answered, during whole video structural information superposition, by the generating process of structured message and structuring
The additive process of information is separated, and arithmetic server need not carry out structured message superposition, structured message additive process by
User terminal is executing, and structured message additive process is shared on multiple user terminals, different user terminal structuring
The superposition of data treating capacity of information also greatly reduces, and improves the stack velocity of structured message, thus both ensure that video flowing
The real-time of display, reduces the requirement to overlapping system software and hardware configuration again.
In addition, the process that the structured message generating arithmetic server is transferred to user terminal can be by algorithm service
Device directly sends structured message to user terminal, can also be and is separately provided a message server, and arithmetic server will
Structured message sends to message server, then is sent structured message to client by message server, and user terminal will
Self-defined information in structured message is superimposed at the position specify in corresponding frame of video, and display in real time is superimposed with and makes by oneself
The Video stream information of adopted information.Same real-time video source, is processed through arithmetic server, will be real for the structured message of every frame output
When be published on message server, user terminal ocx passes through to subscribe to this structured message on message server simultaneously, and synchronous
It is added on corresponding video frame images, thus realizing the superposition of real-time video self-defined information, the superposition of structured message
Journey is to solve synchronicity, real-time and the accuracy of video and overlapped information.
Wherein, above-mentioned arithmetic server, when processing frame of video, is stabbed unique as every frame structure information using frame time
Mark, the accurate location showing in this two field picture containing self-defined information in structured message, in different video frame images,
This position is that there is change, then structured message is published to message server or is sent directly to user terminal；With
When family terminal ocx carries out real-time stream broadcasting, this structured message is subscribed to from message server simultaneously, then according to every frame when
Between stamp is synchronous that each structured message is added on corresponding video frame images, realize the superposition of real-time video self-defined information.
Wherein, after subscription refers to subscribe to certain theme message from message server, message server will send the content under this theme
To booking reader's terminal.
Further, receive frame of video and the speed of structured message to improve user terminal, and improve user eventually
The stack velocity to structured message for the end, based on this, above-mentioned user terminal receives above-mentioned frame of video by different threads respectively
With said structure information, specifically, each frame of video that video source server sends is received by first thread, and by the
Each frame of video corresponding structured message that two thread receiving algorithm servers send, and by the second thread to receiving
Structured message is superimposed at the position specify in corresponding frame of video.
Further it is contemplated that needing the regular hour, for user terminal during message server generating structure information
For, it is poor to there is the regular hour in the frame of video that receives and the corresponding structured message of this frame of video, based on this, above-mentioned user
After terminal receives at least one frame of video, also include: caching process is carried out to each above-mentioned frame of video, and one by one to caching after
Above-mentioned frame of video be decoded and surface rendering is processed, according to the sequencing after output surface renders frame by frame of surface rendering
Above-mentioned frame of video；
After above-mentioned user terminal receives the structured message that above-mentioned arithmetic server sends, also include: by receive
Multiple said structure information are sequentially stored in buffer queue.
Specifically, user terminal carries out caching process according to the preset buffer memory time to the frame of video receiving, according to caching
Sequencing export the frame of video after caching frame by frame, one by one to caching after frame of video be decoded process, and to decoding after
Frame of video carry out surface rendering process, according to the frame of video after output surface renders frame by frame for the sequencing of surface rendering；Right
The frame of video after the surface rendering of output is carried out with the superposition of structured message again.
In the embodiment that the present invention provides, the frame of video receiving is carried out with caching process and by the knot receiving
Structure information is stored in buffer queue, then, the timestamp according to frame of video search from buffer queue with surface rendering after
The corresponding structured message of frame of video, therefore, it is possible to solve because the retardance that arithmetic server calculates leads to there may be can not
Exactly structured message is superimposed to the problem of corresponding frame of video it is possible to be further ensured that fluency and the reality of video flowing
Shi Xing.
Specifically, above-mentioned user terminal is sat according to the timestamp of above-mentioned frame of video, above-mentioned video time stamp and above-mentioned position
Above-mentioned self-defined information is superimposed in above-mentioned frame of video mark, comprising:
During frame of video after receiving surface rendering, according to the video in the timestamp and structured message of this frame of video
Timestamp chooses structured message corresponding with this frame of video from above-mentioned buffer queue；
Above-mentioned self-defined information and above-mentioned position coordinateses are extracted from the above in change information；
Above-mentioned self-defined information is superimposed at the above-mentioned position coordinateses in above-mentioned frame of video.
Further, the above-mentioned timestamp according to above-mentioned frame of video and above-mentioned video time stamp are selected from above-mentioned buffer queue
Before taking said structure information corresponding with above-mentioned frame of video, also include:
Judge to whether there is said structure information in above-mentioned buffer queue, if existing, according to being stored in above-mentioned caching team
The sequencing of row chooses a said structure information one by one, extracts above-mentioned video time stamp from the above in change information.
If in addition, not existing, directly display this frame of video.
Wherein, the above-mentioned timestamp according to above-mentioned frame of video and above-mentioned video time stamp choose from above-mentioned buffer queue with
The corresponding said structure information of above-mentioned frame of video, including:
Timestamp according to above-mentioned video time stamp and above-mentioned frame of video determines the processing mode to said structure information；
Using above-mentioned processing mode, said structure information is processed accordingly, above-mentioned processing mode includes: will be upper
State structured message and return above-mentioned buffer queue as structured message corresponding with above-mentioned frame of video, by said structure information
In or said structure information removed from above-mentioned buffer queue；
When determining that above-mentioned processing mode is to believe said structure information as structuring corresponding with above-mentioned frame of video
Cease or determine that above-mentioned processing mode is by when in the said structure information above-mentioned buffer queue of return, terminate choosing next
The operation of individual said structure information, and choose corresponding structured message for a later frame of above-mentioned frame of video.
Specifically, during frame of video after receiving surface rendering, first, using this frame of video as currently video to be superimposed
Frame, extracts the timestamp of this current frame of video to be superimposed, and judges to whether there is structured message in buffer queue；If not depositing
Then directly displaying this frame of video；If existing, the sequencing according to being stored in buffer queue chooses a structuring letter one by one
Breath is as current operation object, and extracts the video time stamp of this current operation object；Then, according to this current operation object
Video time stamp determines the processing mode to this current operation object with the timestamp of currently frame of video to be superimposed；From unselected
Structured message in choose structured message as current operation object, until determining that this current operation object is should
The corresponding structured message of frame of video or determine that video time stamp in this current operation object is more than currently to be superimposed regarding
The timestamp of frequency frame.
Specifically, the above-mentioned timestamp according to above-mentioned video time stamp and above-mentioned frame of video determines to said structure information
Processing mode, including:
When above-mentioned video time stamp is consistent with the timestamp of above-mentioned frame of video, determine the process to said structure information
Mode is as structured message corresponding with above-mentioned frame of video using said structure information；
When the timestamp of above-mentioned video time stamp and above-mentioned frame of video is inconsistent, judge whether above-mentioned video time stamp is big
In the timestamp of above-mentioned frame of video, if it is determined that the processing mode to said structure information is by said structure information
Return in above-mentioned buffer queue, if not it is determined that to the processing mode of said structure information be by said structure information from
Remove in above-mentioned buffer queue.
Wherein it is considered to when there being multiple structured message in buffer queue, according to be stored in the sequencing of buffer queue according to
Secondary taking-up structured message, between the timestamp of the video time stamp in comparative structure information and frame of video currently to be superimposed
Magnitude relationship, main include following several situations:
(1) when the timestamp of the video time stamp in structured message and frame of video currently to be superimposed is equal, should
Structured message, as the corresponding structured message of this frame of video, terminates choosing next structured message from buffer queue,
Using a later frame of this frame of video as next frame of video to be superimposed；
(2) when the video time stamp in structured message is more than the timestamp of frame of video currently to be superimposed, this is tied
Structure information returns in above-mentioned buffer queue, terminates choosing next structured message from buffer queue, by this frame of video
A later frame is as next frame of video to be superimposed；
(3) when the video time stamp in structured message is less than the timestamp of frame of video currently to be superimposed, this is tied
Structure information is removed from above-mentioned buffer queue, continues to choose next structured message from buffer queue, compares this structure
Magnitude relationship between the timestamp of the video time stamp in change information and frame of video currently to be superimposed, until structured message
In video time stamp and frame of video currently to be superimposed timestamp is equal, the video time stamp in structured message is more than and works as
It is empty in the timestamp of front frame of video to be superimposed or buffer queue.
In the embodiment that the present invention provides, by the sequencing that is stored according to structured message in buffer queue successively
Choose a structured message, the timestamp of the video time stamp in comparative structure information and frame of video currently to be superimposed it
Between magnitude relationship, processing mode to this structured message is determined according to this magnitude relationship, and then realizes quickly and accurately to
Structured message is superimposed in corresponding frame of video.
Specifically, as shown in Fig. 2 giving the schematic flow sheet of the stacking method of preferred video structural information, tool
Body includes:
S10: user terminal receives each frame of video that video source server sends, that is, pass through first thread and take from video source
Business device pulls live video stream；
S11: user terminal carries out caching process according to default cache-time to the frame of video receiving, e.g., to video
Frame carries out 2 seconds cachings, and wherein, this cache-time 2 seconds is that process time consumption determines during whole flow process；
S20: meanwhile, the structured message of user terminal receiving algorithm server or message server transmission, that is, pass through
Second thread subscribes to the structured message of each frame of video from arithmetic server or message server；
S21: and persistently the structured message receiving is stored in buffer queue；
S12: and then, user terminal also by first thread to caching after frame of video be decoded and surface rendering at
Reason, each frame of video that will cache is sequentially sent to decoder module, frame by frame output decoding frame data；After exporting a frame yuv data,
Carry out internal memory (backstage) surface rendering；
Wherein, internal memory (backstage) surface rendering refers to double buffering technology, and a kind of video shows common technology, that is, in internal memory
Create an object consistent with screen drawing region, first by this object in internal memory of graphic plotting, property general again
Picture copying on this object is on screen, thus greatly speeding up the speed of drawing；
S13: user terminal whether there is structured message also by first thread detection buffer queue；
S14: when there is structured message in buffer queue, choose one by one from structured message buffer queue queue
One structured message, to determine and each frame of video yuv pts identical structured message element；
Wherein, it is generally all yuv data form after video frame decoding, pts refers to presentation time stamp,
It is mainly used in measuring when decoded frame of video is revealed, yuv pts refers to the timestamp of this yuv frame of video；
S15: judge that the video time stamp in the structured message chosen with the timestamp of frame of video currently to be superimposed is
No consistent；
S16: when the video time stamp in structured message is consistent with the timestamp of frame of video currently to be superimposed, should
Structured message is as the corresponding structured message of this frame of video, and this structured message is superimposed to video currently to be superimposed
In frame, terminate choosing next structured message from buffer queue, a later frame of this frame of video is to be superimposed as the next one
Frame of video；
S17: when the timestamp of the video time stamp in structured message and frame of video currently to be superimposed is inconsistent, sentence
Whether the video time stamp in disconnected structured message is more than the timestamp of frame of video currently to be superimposed；
S18: when the video time stamp in structured message is more than the timestamp of frame of video currently to be superimposed, this is tied
Structure information returns in above-mentioned buffer queue, terminates choosing next structured message from buffer queue, by this frame of video
A later frame is as next frame of video to be superimposed；
S19: when the video time stamp in structured message is less than the timestamp of frame of video currently to be superimposed, this is tied
Structure information is removed from above-mentioned buffer queue, execution step s14: continues to choose next structuring letter from buffer queue
Breath, judges whether the video time stamp in this structured message is consistent with the timestamp of frame of video currently to be superimposed, Zhi Daojie
Video time stamp in structure information is consistent with the timestamp of frame of video currently to be superimposed, the video time in structured message
It is empty that stamp is more than in the timestamp of frame of video currently to be superimposed or buffer queue；
S110: when there is not structured message in buffer queue queue or be superimposed to structured message corresponding
When in frame of video, directly internal memory render surfaces flip is shown to front end form or structured message has been superimposed to correspondence
Frame of video show to front end form, wherein, flip refer to exchange the meaning, internal memory render surfaces are exchanged to foreground surface
(representing video card equipment).
In the stacking method of the video structural information that the present invention provides, first, video source server is with frame of video shape
To arithmetic server and user terminal synchronous transfer live video stream, arithmetic server is only used for generating each frame of video correspondence formula
Structured message, then, user terminal receives video flowing and the corresponding structured message of each frame of video simultaneously, and by each
Structured message is superimposed in corresponding frame of video, during whole video structural information superposition, by structured message
Generating process and the additive process of structured message separate, arithmetic server need not carry out structured message superposition, knot
Structure information superposition process to be executed by user terminal, and structured message additive process is shared on multiple user terminals, no
The superposition of data treating capacity of same user terminal structured message also greatly reduces, and improves the stack velocity of structured message,
Thus both ensure that the real-time that video flowing shows, reduce the requirement to overlapping system software and hardware configuration again；Further, right
The frame of video receiving carries out caching process and the structured message receiving is stored in buffer queue, then, according to regarding
The timestamp of frequency frame searches structured message corresponding with the frame of video after surface rendering from buffer queue, therefore, it is possible to solution
Certainly because the retardance that arithmetic server calculates leads to there may be and exactly structured message can not be superimposed to corresponding video
The problem of frame is it is possible to be further ensured that fluency and the real-time of video flowing；By being stored in caching team according to structured message
Sequencing in row chooses a structured message successively, the video time stamp in comparative structure information and currently to be superimposed
The timestamp of frame of video between magnitude relationship, processing mode to this structured message is determined according to this magnitude relationship, enters
And realize quickly and accurately structured message being superimposed in corresponding frame of video.
The embodiment of the present invention also provides a kind of user terminal, as shown in figure 3, this user terminal includes:
Frame of video receiver module 202, for receiving at least one frame of video, this frame of video is sent out for video source server sync
Give above-mentioned user terminal and arithmetic server；
Structured message receiver module 204, for receiving the structured message that above-mentioned arithmetic server sends, this structuring
Information is monitoring objective that above-mentioned arithmetic server identifies in above-mentioned frame of video and is generated according to above-mentioned monitoring objective, this structure
Change information includes the position coordinateses of video time stamp, the corresponding self-defined information of monitoring objective and this self-defined information；
Information superposition module 206, for sitting according to the timestamp of above-mentioned frame of video, above-mentioned video time stamp and above-mentioned position
Above-mentioned self-defined information is superimposed in above-mentioned frame of video mark.
In the embodiment that the present invention provides, first, video source server is with frame of video form to arithmetic server and use
Family terminal synchronous transfer live video stream, arithmetic server is only used for generating the corresponding structured message of each frame of video, then,
User terminal receives video flowing and the corresponding structured message of each frame of video simultaneously, and each structured message is superimposed to phase
In the frame of video answered, during whole video structural information superposition, by the generating process of structured message and structuring
The additive process of information is separated, and arithmetic server need not carry out structured message superposition, structured message additive process by
User terminal is executing, and structured message additive process is shared on multiple user terminals, different user terminal structuring
The superposition of data treating capacity of information also greatly reduces, and improves the stack velocity of structured message, thus both ensure that video flowing
The real-time of display, reduces the requirement to overlapping system software and hardware configuration again.
Further, receive frame of video and the speed of structured message to improve user terminal, and improve user eventually
The stack velocity to structured message for the end, based on this, above-mentioned user terminal receives above-mentioned frame of video by different threads respectively
With said structure information.
Further it is contemplated that needing the regular hour, for user terminal during message server generating structure information
For, it is poor to there is the regular hour in the frame of video that receives and the corresponding structured message of this frame of video, based on this, above-mentioned user
Terminal also includes:
Frame of video processing module, for carrying out caching process, and one by one to above-mentioned after caching to each above-mentioned frame of video
Frame of video is decoded and surface rendering is processed, and above-mentioned after output surface renders frame by frame of sequencing according to surface rendering regards
Frequency frame；
Structured message cache module, for being sequentially stored into buffer queue by the multiple said structure information receiving
In.
Further, above- mentioned information laminating module 206 includes: chooses unit, for the timestamp according to above-mentioned frame of video
Choose said structure information corresponding with above-mentioned frame of video with above-mentioned video time stamp from above-mentioned buffer queue；
Extraction unit, extracts above-mentioned self-defined information and above-mentioned position coordinateses in change information from the above；
Superpositing unit, at the above-mentioned position coordinateses that are superimposed to above-mentioned self-defined information in above-mentioned frame of video.
Further, above- mentioned information laminating module 206 also includes: judging unit, for judging in above-mentioned buffer queue to be
No have said structure information, if exist, the sequencing according to being stored in above-mentioned buffer queue choose one by one one above-mentioned
Structured message, extracts above-mentioned video time stamp in change information from the above.
Further, above-mentioned selection unit includes: processing mode determination subelement, for according to above-mentioned video time stamp and
The timestamp of above-mentioned frame of video determines the processing mode to said structure information；
Structured message chooses subelement, for being located accordingly to said structure information using above-mentioned processing mode
Reason, above-mentioned processing mode includes: using said structure information as structured message corresponding with above-mentioned frame of video, by above-mentioned knot
Structure information returns in above-mentioned buffer queue or removes said structure information from above-mentioned buffer queue；When determining
Above-mentioned processing mode is as structured message corresponding with above-mentioned frame of video or to determine above-mentioned using said structure information
When processing mode is to return said structure information in above-mentioned buffer queue, terminate choosing next said structure information
Operation, and choose corresponding structured message for a later frame of above-mentioned frame of video.
Further, above-mentioned processing mode determination subelement, specifically for when above-mentioned video time stamp and above-mentioned frame of video
Timestamp consistent when, determine to the processing mode of said structure information be using said structure information as with above-mentioned video
The corresponding structured message of frame；When the timestamp of above-mentioned video time stamp and above-mentioned frame of video is inconsistent, judge above-mentioned video
Whether timestamp is more than the timestamp of above-mentioned frame of video, if it is determined that the processing mode to said structure information is will be upper
State structured message and return in above-mentioned buffer queue, if not it is determined that the processing mode to said structure information is will be above-mentioned
Structured message is removed from above-mentioned buffer queue.
In the user terminal that the present invention provides, user terminal receives video flowing and the corresponding structure of each frame of video simultaneously
Change information, and each structured message is superimposed in corresponding frame of video, structured message additive process is come by user terminal
Execution, and structured message additive process is shared on multiple user terminals, the superposition of different user terminal structured message
Data processing amount also greatly reduces, and improves the stack velocity of structured message, thus it is real-time both to have ensure that video flowing showed
Property, reduce the requirement to overlapping system software and hardware configuration again；Further, the frame of video receiving is carried out caching process,
And the structured message receiving is stored in buffer queue, then, looked into from buffer queue according to the timestamp of frame of video
Look for structured message corresponding with the frame of video after surface rendering, therefore, it is possible to solve the retardance calculating because of arithmetic server
Lead to there may be the problem that exactly structured message can not be superimposed to corresponding frame of video and regard it is possible to be further ensured that
The fluency of frequency stream and real-time；One knot is chosen successively by the sequencing being stored in buffer queue according to structured message
Structure information, the size between the timestamp of the video time stamp in comparative structure information and frame of video currently to be superimposed is closed
System, determines the processing mode to this structured message according to this magnitude relationship, and then realizes quickly and accurately by structured message
It is superimposed in corresponding frame of video.
The embodiment of the present invention also provides a kind of overlapping system of video structural information, as shown in figure 4, this system includes:
Video source server 11, arithmetic server 22 and user terminal 33, above-mentioned video source server 11 respectively with above-mentioned algorithm service
Device 22 and above-mentioned user terminal 33 communicate to connect, and above-mentioned user terminal 33 is communicated to connect with above-mentioned arithmetic server 22；
Above-mentioned video source server 11, for being sent to above-mentioned user terminal 33 and above-mentioned by least one video frame synchronization
Arithmetic server 22；
Above-mentioned arithmetic server 22, for receiving above-mentioned frame of video, and identifies the monitoring objective in above-mentioned frame of video, according to
Above-mentioned monitoring objective generates the corresponding structured message of above-mentioned frame of video, by said structure information transfer to above-mentioned user terminal
33, said structure information includes video time stamp, the corresponding self-defined information of above-mentioned monitoring objective and above-mentioned self-defined letter
The position coordinateses of breath；
Above-mentioned user terminal 33 is used for receiving the structured message that above-mentioned frame of video and above-mentioned arithmetic server 22 send, and
According to the timestamp of above-mentioned frame of video, above-mentioned video time stamp and above-mentioned position coordinateses, above-mentioned self-defined information is superimposed to above-mentioned
In frame of video.
Further, as shown in figure 5, above-mentioned overlapping system can also include: whole with arithmetic server 22 and user respectively
The message server 44 of end 33 communication connection, wherein, arithmetic server 22 sends structured message to message server 44, then
By message server 44, structured message is sent to client, the self-defined information in structured message is folded by user terminal 33
Add at the position specified in corresponding frame of video, and display is superimposed with the Video stream information of self-defined information in real time.Same reality
When video source, process through arithmetic server 22, by every frame output structured message real-time release on message server 44,
User terminal 33ocx passes through to subscribe to this structured message on message server 44 simultaneously, and the synchronous corresponding video that is added to
On two field picture, thus realizing the superposition of real-time video self-defined information.
Understood based on above-mentioned analysis, in the overlapping system of video structural information provided in an embodiment of the present invention, video source
Server 11 is with frame of video form to arithmetic server 22 and user terminal 33 synchronous transfer live video stream, arithmetic server 22
It is only used for generating the corresponding structured message of each frame of video, then, user terminal 33 receives video flowing and each video simultaneously
The corresponding structured message of frame, and each structured message is superimposed in corresponding frame of video, in whole video structural letter
During breath superposition, the additive process of the generating process of structured message and structured message is separated, algorithm service
Device 22 need not carry out structured message superposition, and structured message additive process to be executed by user terminal 33, and structured message
Additive process is shared on multiple user terminals 33, and the superposition of data treating capacity of different user terminal 33 structured messages is also big
Big minimizing, improves the stack velocity of structured message, thus both ensure that the real-time that video flowing shows, reduces to folded again
The requirement of adding system software and hardware configuration.
The there is provided user terminal of the embodiment of the present invention for the specific hardware on equipment or can be installed on soft on equipment
Part or firmware etc..User terminal and overlapping system that the embodiment of the present invention is provided, it realizes the technique effect of principle and generation
Identical with preceding method embodiment, for briefly describing, user terminal and overlapping system embodiment part do not refer to part, refer to
Corresponding contents in preceding method embodiment.Those skilled in the art can be understood that, for description convenience and letter
Clean, the specific work process of system, user terminal and unit described above, it is right in said method embodiment all to may be referred to
Answer process, will not be described here.
It should be understood that disclosed user terminal and method in embodiment provided by the present invention, other can be passed through
Mode realize.User terminal embodiment described above is only schematically, for example, the division of described unit, and only
For a kind of division of logic function, actual can have other dividing mode when realizing, and for example, multiple units or assembly can be tied
Close or be desirably integrated into another system, or some features can be ignored, or do not execute.Another, shown or discussed
Coupling each other or direct-coupling or communication connection can be by some communication interfaces, user terminal or unit indirect
Coupling or communication connection, can be electrical, mechanical or other forms.
The described unit illustrating as separating component can be or may not be physically separate, show as unit
The part showing can be or may not be physical location, you can with positioned at a place, or can also be distributed to multiple
On NE.The mesh to realize this embodiment scheme for some or all of unit therein can be selected according to the actual needs
's.
In addition, each functional unit in the embodiment that the present invention provides can be integrated in a processing unit, also may be used
To be that unit is individually physically present it is also possible to two or more units are integrated in a unit.
If described function realized using in the form of SFU software functional unit and as independent production marketing or use when, permissible
It is stored in a computer read/write memory medium.Based on such understanding, technical scheme is substantially in other words
Partly being embodied in the form of software product of part that prior art is contributed or this technical scheme, this meter
Calculation machine software product is stored in a storage medium, including some instructions with so that a computer equipment (can be individual
People's computer, server, or network equipment etc.) execution each embodiment methods described of the present invention all or part of step.
And aforesaid storage medium includes: u disk, portable hard drive, read only memory (read-only memory, rom), random access memory are deposited
Reservoir (random access memory, ram), magnetic disc or CD etc. are various can be with the medium of store program codes.
It should also be noted that similar label and letter expression similar terms in following accompanying drawing, therefore, once a certain Xiang Yi
It is defined in individual accompanying drawing, then do not need it to be defined further and explains in subsequent accompanying drawing, additionally, term " the
One ", " second ", " the 3rd " etc. are only used for distinguishing description, and it is not intended that indicating or hint relative importance.
Last it is noted that the specific embodiment of embodiment described above, the only present invention, in order to the present invention to be described
Technical scheme, be not intended to limit, protection scope of the present invention is not limited thereto, although with reference to the foregoing embodiments to this
Bright be described in detail, it will be understood by those within the art that: any those familiar with the art
The invention discloses technical scope in, it still can be modified to the technical scheme described in previous embodiment or can be light
It is readily conceivable that change, or equivalent is carried out to wherein some technical characteristics；And these modifications, change or replacement, do not make
The essence of appropriate technical solution departs from the spirit and scope of embodiment of the present invention technical scheme.The protection in the present invention all should be covered
Within the scope of.Therefore, protection scope of the present invention should described be defined by scope of the claims.
Claims (15)
1. a kind of stacking method of video structural information is it is characterised in that methods described includes:
User terminal receives at least one frame of video, and described frame of video is sent to described user terminal for video source server sync
With arithmetic server；
Described user terminal receives the structured message that described arithmetic server sends, and described structured message is described algorithm clothes
Business device identify described frame of video in monitoring objective and according to described monitoring objective generate, described structured message includes video
The position coordinateses of timestamp, the corresponding self-defined information of described monitoring objective and described self-defined information；
Described user terminal is made by oneself described according to the timestamp of described frame of video, described video time stamp and described position coordinateses
Adopted information superposition is to described frame of video.
2. method according to claim 1 receives institute it is characterised in that described user terminal passes through different threads respectively
State frame of video and described structured message.
3. method according to claim 1 is it is characterised in that after described user terminal receives at least one frame of video,
Also include: frame of video each described is carried out with caching process, and one by one the described frame of video after caching is decoded and surface
Render process, according to the described frame of video after output surface renders frame by frame for the sequencing of surface rendering；
After described user terminal receives the structured message that described arithmetic server sends, also include: multiple by receive
Described structured message is sequentially stored in buffer queue.
4. method according to claim 3 it is characterised in that described user terminal according to the timestamp of described frame of video,
Described self-defined information is superimposed in described frame of video for described video time stamp and described position coordinateses, comprising:
Timestamp according to described frame of video and described video time stamp are chosen and described frame of video pair from described buffer queue
The described structured message answered；
Described self-defined information and described position coordinateses are extracted from described structured message；
Described self-defined information is superimposed at the described position coordinateses in described frame of video.
5. method according to claim 4 is it is characterised in that the described timestamp according to described frame of video and described video
Before timestamp chooses described structured message corresponding with described frame of video from described buffer queue, also include:
Judge to whether there is described structured message in described buffer queue, if existing, according to being stored in described buffer queue
Sequencing chooses a described structured message one by one, extracts described video time stamp from described structured message.
6. method according to claim 5 is it is characterised in that the described timestamp according to described frame of video and described video
Timestamp chooses described structured message corresponding with described frame of video from described buffer queue, including:
Timestamp according to described video time stamp and described frame of video determines the processing mode to described structured message；
Using described processing mode, described structured message is processed accordingly, described processing mode includes: by described knot
Structure information as structured message corresponding with described frame of video, by described structured message return described buffer queue in,
Or described structured message is removed from described buffer queue；
When determine described processing mode be using described structured message as structured message corresponding with described frame of video or
Person determines that described processing mode is by when in the described structured message described buffer queue of return, terminates choosing described in the next one
The operation of structured message, and choose corresponding structured message for a later frame of described frame of video.
7. method according to claim 6 it is characterised in that described according to described video time stamp and described frame of video
Timestamp determines the processing mode to described structured message, including:
When described video time stamp is consistent with the timestamp of described frame of video, determine the processing mode to described structured message
It is as structured message corresponding with described frame of video using described structured message；
When the timestamp of described video time stamp and described frame of video is inconsistent, judge whether described video time stamp is more than institute
State the timestamp of frame of video, if it is determined that the processing mode to described structured message is to return described structured message
In described buffer queue, if not it is determined that the processing mode to described structured message is from described by described structured message
Remove in buffer queue.
8. a kind of user terminal is it is characterised in that described user terminal includes:
Frame of video receiver module, for receiving at least one frame of video, described frame of video is sent to for video source server sync
Described user terminal and arithmetic server；
Structured message receiver module, for receiving the structured message that described arithmetic server sends, described structured message
For the monitoring objective in the described arithmetic server described frame of video of identification and according to the generation of described monitoring objective, described structuring
Information includes the position coordinateses of video time stamp, the corresponding self-defined information of described monitoring objective and described self-defined information；
Information superposition module, for according to the timestamp of described frame of video, described video time stamp and described position coordinateses by institute
State self-defined information to be superimposed in described frame of video.
9. user terminal according to claim 8 is it is characterised in that described user terminal is connect by different threads respectively
Receive described frame of video and described structured message.
10. user terminal according to claim 8 is it is characterised in that described user terminal also includes:
Frame of video processing module, for carrying out caching process, and one by one to the described video after caching to frame of video each described
Frame is decoded and surface rendering is processed, according to the described video after output surface renders frame by frame for the sequencing of surface rendering
Frame；
Structured message cache module, for being sequentially stored into the multiple described structured message receiving in buffer queue.
11. user terminals according to claim 10 are it is characterised in that described information laminating module includes:
Choose unit, chooses from described buffer queue for the timestamp according to described frame of video and described video time stamp and
The corresponding described structured message of described frame of video；
Extraction unit, for extracting described self-defined information and described position coordinateses from described structured message；
Superpositing unit, at the described position coordinateses that are superimposed to described self-defined information in described frame of video.
12. user terminals according to claim 11 are it is characterised in that described information laminating module also includes:
Judging unit, for judging to whether there is described structured message in described buffer queue, if exist, according to being stored in
The sequencing stating buffer queue chooses a described structured message one by one, extracts described video from described structured message
Timestamp.
13. user terminals according to claim 12 are it is characterised in that described selection unit includes:
Processing mode determination subelement, determines to described knot for the timestamp according to described video time stamp and described frame of video
The processing mode of structure information；
Structured message chooses subelement, for being processed accordingly to described structured message using described processing mode,
Described processing mode includes: using described structured message as structured message corresponding with described frame of video, by described structure
Change information returns in described buffer queue or removes described structured message from described buffer queue；When determining
Stating processing mode is that as structured message corresponding with described frame of video or described structured message is determined described place
When reason mode is to return described structured message in described buffer queue, terminate choosing the behaviour of next described structured message
Make, and choose corresponding structured message for a later frame of described frame of video.
14. user terminals according to claim 13, it is characterised in that described processing mode determination subelement, are specifically used
In when described video time stamp is consistent with the timestamp of described frame of video, determine that the processing mode to described structured message is
Using described structured message as structured message corresponding with described frame of video；When described video time stamp and described frame of video
Timestamp inconsistent when, judge described video time stamp whether more than the timestamp of described frame of video, if it is determined that to institute
The processing mode stating structured message is to return in described buffer queue by described structured message, if not it is determined that to described
The processing mode of structured message is to remove described structured message from described buffer queue.
A kind of 15. overlapping systems of video structural information are it is characterised in that described system includes: video source server, algorithm
Server and user terminal, described video source server is connected with described arithmetic server and described user terminal communication respectively,
Described user terminal and described arithmetic server communication connection；
Described video source server, for being sent to described user terminal and described algorithm service by least one video frame synchronization
Device；
Described arithmetic server, for receiving described frame of video, and identifies the monitoring objective in described frame of video, according to described prison
Survey target and generate the corresponding structured message of described frame of video, described structured message is transmitted to described user terminal, described
Structured message includes the position of video time stamp, the corresponding self-defined information of described monitoring objective and described self-defined information
Put coordinate；
Described user terminal is used for receiving the structured message that described frame of video and described arithmetic server send, and according to described
Described self-defined information is superimposed to described frame of video by the timestamp of frame of video, described video time stamp and described position coordinateses
In.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610763495.2A CN106375793B (en) | 2016-08-29 | 2016-08-29 | video structured information superposition method, user terminal and superposition system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610763495.2A CN106375793B (en) | 2016-08-29 | 2016-08-29 | video structured information superposition method, user terminal and superposition system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106375793A true CN106375793A (en) | 2017-02-01 |
| CN106375793B CN106375793B (en) | 2019-12-13 |
Family
ID=57901445
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610763495.2A Active CN106375793B (en) | 2016-08-29 | 2016-08-29 | video structured information superposition method, user terminal and superposition system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106375793B (en) |
Cited By (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107832402A (en) * | 2017-11-01 | 2018-03-23 | 武汉烽火众智数字技术有限责任公司 | Dynamic exhibition system and its method during a kind of video structural fructufy |
| CN109309817A (en) * | 2018-11-14 | 2019-02-05 | 北京东方国信科技股份有限公司 | The display methods and device of monitor video recognition of face OSD |
| CN109862396A (en) * | 2019-01-11 | 2019-06-07 | 苏州科达科技股份有限公司 | A kind of analysis method of video code flow, electronic equipment and readable storage medium storing program for executing |
| CN110087042A (en) * | 2019-05-08 | 2019-08-02 | 深圳英飞拓智能技术有限公司 | A kind of face snap method and system of real-time synchronization video flowing and metadata |
| CN110647173A (en) * | 2019-10-10 | 2020-01-03 | 四川赛狄信息技术股份公司 | Video tracking system and method |
| CN110874424A (en) * | 2019-09-23 | 2020-03-10 | 北京旷视科技有限公司 | Data processing method and device, computer equipment and readable storage medium |
| CN111064984A (en) * | 2018-10-16 | 2020-04-24 | 杭州海康威视数字技术股份有限公司 | Intelligent information superposition display method and device for video frame and hard disk video recorder |
| WO2020097857A1 (en) * | 2018-11-15 | 2020-05-22 | 北京比特大陆科技有限公司 | Media stream processing method and apparatus, storage medium, and program product |
| CN111294666A (en) * | 2019-07-04 | 2020-06-16 | 杭州萤石软件有限公司 | Video frame transmission method and method, device and system for determining video frame transmission delay |
| CN111629264A (en) * | 2020-06-01 | 2020-09-04 | 复旦大学 | Web-based separate front-end image rendering method |
| CN111698546A (en) * | 2020-06-29 | 2020-09-22 | 平安国际智慧城市科技股份有限公司 | Video structured result transmission method and device, terminal equipment and storage medium |
| CN111832366A (en) * | 2019-04-22 | 2020-10-27 | 鸿富锦精密电子(天津)有限公司 | Image recognition device and method |
| CN112203113A (en) * | 2020-12-07 | 2021-01-08 | 北京沃东天骏信息技术有限公司 | Video stream structuring method and device, electronic equipment and computer readable medium |
| CN112235598A (en) * | 2020-09-27 | 2021-01-15 | 深圳云天励飞技术股份有限公司 | Video structured processing method and device and terminal equipment |
| CN112584073A (en) * | 2020-12-24 | 2021-03-30 | 杭州叙简科技股份有限公司 | 5G-based law enforcement recorder distributed assistance calculation method |
| CN112950951A (en) * | 2021-01-29 | 2021-06-11 | 浙江大华技术股份有限公司 | Intelligent information display method, electronic device and storage medium |
| CN113923507A (en) * | 2021-12-13 | 2022-01-11 | 北京蔚领时代科技有限公司 | Low-delay video rendering method and device for Android terminal |
| CN114051120A (en) * | 2021-10-26 | 2022-02-15 | 远光软件股份有限公司 | Video alarm method, device, storage medium and electronic equipment |
| CN114900727A (en) * | 2022-05-11 | 2022-08-12 | 上海哔哩哔哩科技有限公司 | Video stream processing method and device |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1852431A (en) * | 2005-12-23 | 2006-10-25 | 华为技术有限公司 | System and method for realizing video frequency information sharing |
| CN101593346A (en) * | 2009-07-06 | 2009-12-02 | 中国人民解放军总装备部军械技术研究所 | Integrated general target video image acquisition, recognition and tracking device |
| CN102647618A (en) * | 2012-04-28 | 2012-08-22 | 深圳市华鼎视数字移动电视有限公司 | Method and system for interaction with television programs |
| CN102957966A (en) * | 2011-08-19 | 2013-03-06 | 深圳市快播科技有限公司 | Player and method for embedding time in video frames of player |
| CN103635954A (en) * | 2011-02-08 | 2014-03-12 | 隆沙有限公司 | A system to augment a visual data stream based on geographical and visual information |
| CN103646546A (en) * | 2013-11-23 | 2014-03-19 | 安徽蓝盾光电子股份有限公司 | An intelligent traffic system with a large-scale vehicle passing-forbidding function |
| CN104113727A (en) * | 2013-04-17 | 2014-10-22 | 华为技术有限公司 | Monitoring video playing method, device and system |
| CN105072460A (en) * | 2015-07-15 | 2015-11-18 | 中国科学技术大学先进技术研究院 | Information annotation and association method, system and device based on VCE |
| CN105812722A (en) * | 2014-12-30 | 2016-07-27 | 航天信息股份有限公司 | Grain transportation monitoring method and system |
-
2016
- 2016-08-29 CN CN201610763495.2A patent/CN106375793B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1852431A (en) * | 2005-12-23 | 2006-10-25 | 华为技术有限公司 | System and method for realizing video frequency information sharing |
| CN101593346A (en) * | 2009-07-06 | 2009-12-02 | 中国人民解放军总装备部军械技术研究所 | Integrated general target video image acquisition, recognition and tracking device |
| CN103635954A (en) * | 2011-02-08 | 2014-03-12 | 隆沙有限公司 | A system to augment a visual data stream based on geographical and visual information |
| CN102957966A (en) * | 2011-08-19 | 2013-03-06 | 深圳市快播科技有限公司 | Player and method for embedding time in video frames of player |
| CN102647618A (en) * | 2012-04-28 | 2012-08-22 | 深圳市华鼎视数字移动电视有限公司 | Method and system for interaction with television programs |
| CN104113727A (en) * | 2013-04-17 | 2014-10-22 | 华为技术有限公司 | Monitoring video playing method, device and system |
| CN103646546A (en) * | 2013-11-23 | 2014-03-19 | 安徽蓝盾光电子股份有限公司 | An intelligent traffic system with a large-scale vehicle passing-forbidding function |
| CN105812722A (en) * | 2014-12-30 | 2016-07-27 | 航天信息股份有限公司 | Grain transportation monitoring method and system |
| CN105072460A (en) * | 2015-07-15 | 2015-11-18 | 中国科学技术大学先进技术研究院 | Information annotation and association method, system and device based on VCE |
Cited By (32)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107832402A (en) * | 2017-11-01 | 2018-03-23 | 武汉烽火众智数字技术有限责任公司 | Dynamic exhibition system and its method during a kind of video structural fructufy |
| CN107832402B (en) * | 2017-11-01 | 2021-08-03 | 武汉烽火众智数字技术有限责任公司 | Dynamic display system and method for video structured fruit bearing |
| CN111064984B (en) * | 2018-10-16 | 2022-02-08 | 杭州海康威视数字技术股份有限公司 | Intelligent information superposition display method and device for video frame and hard disk video recorder |
| CN111064984A (en) * | 2018-10-16 | 2020-04-24 | 杭州海康威视数字技术股份有限公司 | Intelligent information superposition display method and device for video frame and hard disk video recorder |
| CN109309817A (en) * | 2018-11-14 | 2019-02-05 | 北京东方国信科技股份有限公司 | The display methods and device of monitor video recognition of face OSD |
| CN112930687B (en) * | 2018-11-15 | 2023-04-28 | 北京比特大陆科技有限公司 | Media stream processing method and device, storage medium and program product |
| WO2020097857A1 (en) * | 2018-11-15 | 2020-05-22 | 北京比特大陆科技有限公司 | Media stream processing method and apparatus, storage medium, and program product |
| CN112930687A (en) * | 2018-11-15 | 2021-06-08 | 北京比特大陆科技有限公司 | Media stream processing method and device, storage medium and program product |
| CN109862396A (en) * | 2019-01-11 | 2019-06-07 | 苏州科达科技股份有限公司 | A kind of analysis method of video code flow, electronic equipment and readable storage medium storing program for executing |
| CN111832366B (en) * | 2019-04-22 | 2024-04-02 | 富联精密电子(天津)有限公司 | Image recognition apparatus and method |
| CN111832366A (en) * | 2019-04-22 | 2020-10-27 | 鸿富锦精密电子(天津)有限公司 | Image recognition device and method |
| CN110087042A (en) * | 2019-05-08 | 2019-08-02 | 深圳英飞拓智能技术有限公司 | A kind of face snap method and system of real-time synchronization video flowing and metadata |
| CN110087042B (en) * | 2019-05-08 | 2021-07-09 | 深圳英飞拓智能技术有限公司 | Face snapshot method and system for synchronizing video stream and metadata in real time |
| CN111294666A (en) * | 2019-07-04 | 2020-06-16 | 杭州萤石软件有限公司 | Video frame transmission method and method, device and system for determining video frame transmission delay |
| CN111294666B (en) * | 2019-07-04 | 2022-07-01 | 杭州萤石软件有限公司 | Video frame transmission method and method, device and system for determining video frame transmission delay |
| CN110874424A (en) * | 2019-09-23 | 2020-03-10 | 北京旷视科技有限公司 | Data processing method and device, computer equipment and readable storage medium |
| CN110647173A (en) * | 2019-10-10 | 2020-01-03 | 四川赛狄信息技术股份公司 | Video tracking system and method |
| CN111629264A (en) * | 2020-06-01 | 2020-09-04 | 复旦大学 | Web-based separate front-end image rendering method |
| CN111629264B (en) * | 2020-06-01 | 2021-07-27 | 复旦大学 | Web-based separate front-end image rendering method |
| CN111698546B (en) * | 2020-06-29 | 2023-02-03 | 平安国际智慧城市科技股份有限公司 | Video structured result transmission method and device, terminal equipment and storage medium |
| CN111698546A (en) * | 2020-06-29 | 2020-09-22 | 平安国际智慧城市科技股份有限公司 | Video structured result transmission method and device, terminal equipment and storage medium |
| CN112235598A (en) * | 2020-09-27 | 2021-01-15 | 深圳云天励飞技术股份有限公司 | Video structured processing method and device and terminal equipment |
| CN112203113A (en) * | 2020-12-07 | 2021-01-08 | 北京沃东天骏信息技术有限公司 | Video stream structuring method and device, electronic equipment and computer readable medium |
| CN112203113B (en) * | 2020-12-07 | 2021-05-25 | 北京沃东天骏信息技术有限公司 | Video stream structuring method and device, electronic equipment and computer readable medium |
| CN112584073B (en) * | 2020-12-24 | 2022-08-02 | 杭州叙简科技股份有限公司 | 5G-based law enforcement recorder distributed assistance calculation method |
| CN112584073A (en) * | 2020-12-24 | 2021-03-30 | 杭州叙简科技股份有限公司 | 5G-based law enforcement recorder distributed assistance calculation method |
| CN112950951A (en) * | 2021-01-29 | 2021-06-11 | 浙江大华技术股份有限公司 | Intelligent information display method, electronic device and storage medium |
| CN112950951B (en) * | 2021-01-29 | 2023-05-02 | 浙江大华技术股份有限公司 | Intelligent information display method, electronic device and storage medium |
| CN114051120A (en) * | 2021-10-26 | 2022-02-15 | 远光软件股份有限公司 | Video alarm method, device, storage medium and electronic equipment |
| CN113923507A (en) * | 2021-12-13 | 2022-01-11 | 北京蔚领时代科技有限公司 | Low-delay video rendering method and device for Android terminal |
| CN113923507B (en) * | 2021-12-13 | 2022-07-22 | 北京蔚领时代科技有限公司 | Low-delay video rendering method and device for Android terminal |
| CN114900727A (en) * | 2022-05-11 | 2022-08-12 | 上海哔哩哔哩科技有限公司 | Video stream processing method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106375793B (en) | 2019-12-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106375793A (en) | Superposition method and superposition system of video structured information, and user terminal | |
| CN101587431B (en) | Method for realizing multi-screen playing video | |
| CN108495120A (en) | A kind of video frame detection, processing method, apparatus and system | |
| CN108632674A (en) | A kind of playback method and client of panoramic video | |
| CN109284784A (en) | A kind of content auditing model training method and device for live scene video | |
| CN109644296A (en) | A kind of video stream transmission method, relevant device and system | |
| CN101237480B (en) | 3d display multi-screen mobile phone and multi-screen display control method | |
| CN103677701B (en) | The method and system of large-size screen monitors simultaneous display | |
| CN108235116A (en) | Feature propagation method and device, electronic equipment, program and medium | |
| CN108900833A (en) | A kind of combination screen synch detection method, device, system and equipment | |
| CN108229363A (en) | Key frame dispatching method and device, electronic equipment, program and medium | |
| CN105872802A (en) | Video playing method and device | |
| CN106341603A (en) | View finding method for virtual reality environment, device and virtual reality device | |
| CN113778360A (en) | Screen projection method and electronic equipment | |
| CN108304148A (en) | A kind of method and apparatus that multi-screen splicing is shown | |
| CN110009407A (en) | A kind of advertisement sending method, device, advertisement playback terminal and storage medium | |
| CN109218796A (en) | A kind of method and apparatus showing virtual present special efficacy | |
| CN109767273A (en) | A kind of advertisement sending method, device, advertisement playback terminal and storage medium | |
| CN103581662A (en) | Video definition measuring method and system | |
| CN106412718A (en) | Rendering method and device for subtitles in 3D space | |
| CN113542907B (en) | Multimedia data transceiving method, system, processor and player | |
| CN113676692A (en) | Video processing method and device in video conference, electronic equipment and storage medium | |
| CN101299843A (en) | 3D display mobile phone and 3D image display method | |
| CN110248207A (en) | Image presence shows server, methods of exhibiting and recording medium and display systems | |
| CN113766297B (en) | Video processing method, playing terminal and computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| PP01 | Preservation of patent right | ||
| PP01 | Preservation of patent right |
Effective date of registration: 20220726 Granted publication date: 20191213 |