CN106534142A - Method and apparatus for real-time transmission of multimedia data - Google Patents
Method and apparatus for real-time transmission of multimedia data Download PDFInfo
- Publication number
- CN106534142A CN106534142A CN201611058946.9A CN201611058946A CN106534142A CN 106534142 A CN106534142 A CN 106534142A CN 201611058946 A CN201611058946 A CN 201611058946A CN 106534142 A CN106534142 A CN 106534142A
- Authority
- CN
- China
- Prior art keywords
- motion sensing
- sensing control
- sensing
- control packet
- motion
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 42
- 230000005540 biological transmission Effects 0.000 title claims description 22
- 238000012544 monitoring process Methods 0.000 claims abstract description 12
- 230000001360 synchronised effect Effects 0.000 claims description 15
- 230000006698 induction Effects 0.000 claims description 9
- 238000002372 labelling Methods 0.000 claims description 9
- 230000035807 sensation Effects 0.000 claims description 7
- 230000000644 propagated effect Effects 0.000 claims description 6
- 241001269238 Data Species 0.000 claims 1
- 230000008569 process Effects 0.000 abstract description 15
- 238000004088 simulation Methods 0.000 abstract description 15
- 238000005516 engineering process Methods 0.000 abstract description 8
- 230000003238 somatosensory effect Effects 0.000 abstract description 8
- 230000006854 communication Effects 0.000 description 18
- 238000004891 communication Methods 0.000 description 17
- 230000006870 function Effects 0.000 description 11
- 230000000272 proprioceptive effect Effects 0.000 description 9
- 230000002452 interceptive effect Effects 0.000 description 8
- 238000012545 processing Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 4
- 238000003860 storage Methods 0.000 description 4
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000008878 coupling Effects 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 241000894007 species Species 0.000 description 2
- 230000006399 behavior Effects 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000004069 differentiation Effects 0.000 description 1
- 230000005611 electricity Effects 0.000 description 1
- 238000013467 fragmentation Methods 0.000 description 1
- 238000006062 fragmentation reaction Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000001727 in vivo Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
- 238000010792 warming Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/24—Monitoring of processes or resources, e.g. monitoring of server load, available bandwidth, upstream requests
- H04N21/2408—Monitoring of the upstream path of the transmission network, e.g. client requests
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
- H04N21/258—Client or end-user data management, e.g. managing client capabilities, user preferences or demographics, processing of multiple end-users preferences to derive collaborative data
- H04N21/25866—Management of end-user data
- H04N21/25891—Management of end-user data being end-user preferences
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Databases & Information Systems (AREA)
- Computer Graphics (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
- Selective Calling Equipment (AREA)
Abstract
The invention is suitable for the field of the network technology. During a video live broadcast process, instruction monitoring is carried out on a live broadcast terminal; when the live broadcast terminal is monitored to generate a somatosensory control instruction, a somatosensory control data packet is generated according to the somatosensory control instruction and an image frame that is collected currently by the live broadcast terminal is marked; the somatosensory control data packet and the image frame are associated; and the image frame is sent to a user terminal in real time by a video live broadcast platform and the somatosensory control data packet associated with the image frame is sent to the user terminal synchronously by the video live broadcast platform, so that the user terminal outputs the video live broadcast platform to a somatosensory sensing device synchronously while outputting the image frame to a video playing device. Therefore, the user can obtain the experience of somatosensory simulation matching the video live-broadcast content while watching the live-broadcast video, so that the on-line scene interactivity is improved substantially.
Description
Technical field
The invention belongs to networking technology area, more particularly to the live transmission method and device of multi-medium data.
Background technology
Multimedia refers to a kind of man-machine interactive communication for information and communications media for combining two or more media, makes
Media include word, picture, sound, animation and film, and the interaction function provided by formula.With science and technology
Development, multimedia technology can also be had to people except bringing visual experience and audio experience for user by computer
Other body-sensings having are simulated, including the sense having to all human bodies such as sense of touch, power sense, warming, wet sense, wind sense, olfactory sensation
Know simulation, think user bring body-sensing in terms of interactive experience.
Body-sensing sensing principle of realizing be：Motion sensing control signal is sent to the body-sensing sensing device for being attached to body surface,
All kinds of body-sensing feedbacks are carried out to human body with control volume propagated sensation induction device.For example, at virtual reality (Virtual Reality, VR)
When shaking hands with virtual portrait in game, by force-feedback control signal be pointed to hand force transducer produce dynamics size and
Persistent period is controlled, so as to bring with play in the behavior of shaking hands match force feedback experience.However, technical staff exists
Find in R＆D process, prior art at least has following technological deficiency：
It is greatly that user oriented is played online in the streaming media that existing video file has, and existing
Video content provider and all kinds of somatosensory device suppliers each separately provide service, video content is believed with body-sensing sensing control
Number form it is also different, therefore, prior art cannot be the online video real-time matching body-sensing data played, and thus reduce
Online scene it is interactive.
The content of the invention
In view of this, the live transmission method and device of multi-medium data are embodiments provided, it is existing to solve
Technology cannot be the online video real-time matching body-sensing data played, and thus reduce the interactive problem of online scene.
In a first aspect, embodiments providing a kind of real-time sending method of multi-medium data, methods described includes：
During net cast, instruction monitoring is carried out to live terminal；
When listen to the live terminal generate motion sensing control instruct when, according to the motion sensing control instruction generate body-sensing
Control data bag, and the picture frame that live terminal described in labelling is currently collected；
The motion sensing control packet and described image frame are associated；
Described image frame is sent to user terminal in real time by net cast platform, while will associate with described image frame
The motion sensing control packet by the net cast platform synchronized transmission to the user terminal so that user's end
While described image frame is exported to video play device, motion sensing control packet described in synchronism output is filled to body-sensing sensing at end
Put.
It is as the first possible implementation of first aspect, described that body-sensing control is generated according to motion sensing control instruction
Packet processed includes：
The motion sensing control data of M body points position are obtained, the motion sensing control data of each body points position include N types
Body-sensing sensing device control information；
The motion sensing control data of the M body points position for getting are write into the motion sensing control packet；
The M and the N are the integer more than or equal to 1.
With reference to the first possible implementation of first aspect, in second possible implementation, the basis
The motion sensing control instruction generates motion sensing control packet also to be included；
If not getting the control information of the body-sensing sensing device of the jth type of i-th body points position, in the body-sensing
In control data bag, corresponding each data bit of the control information writes preset characters；
The i and the j are the integer more than or equal to 1, and the i is less than or equal to the M, the j be less than or
Equal to the N.
It is with reference to second possible implementation of first aspect, in the third possible implementation, described default
Character is 0.
Second aspect, a kind of real-time dispensing device of multi-medium data, described device include：
Monitoring unit, for, during net cast, carrying out instruction monitoring to live terminal；
Signal generating unit, for when listen to the live terminal generate motion sensing control instruct when, according to the body-sensing control
System instruction generates motion sensing control packet, and the picture frame that live terminal described in labelling is currently collected；
Associative cell, for the motion sensing control packet and described image frame are associated；
Transmitting element, for described image frame is sent to user terminal in real time by net cast platform, while will be with
Described image frame association the motion sensing control packet by the net cast platform synchronized transmission to the user terminal,
So that the user terminal is while described image frame is exported to video play device, motion sensing control data described in synchronism output
Wrap to body-sensing sensing device.
Used as the first possible implementation of second aspect, the signal generating unit includes：
Subelement is obtained, for obtaining the motion sensing control data of M body points position, the motion sensing control number of each body points position
According to the control information of the body-sensing sensing device comprising N types；
First write subelement, it is described for the motion sensing control data of the M body points position for getting are write
Motion sensing control packet；
The M and the N are the integer more than or equal to 1.
With reference to the first possible implementation of second aspect, in second possible implementation, the generation
Unit also includes；
Second write subelement, if the body-sensing sensing device of the jth type for not getting i-th body points position
Control information, in the motion sensing control packet, corresponding each data bit of the control information writes preset characters；
The i and the j are the integer more than or equal to 1, and the i is less than or equal to the M, the j be less than or
Equal to the N.
It is with reference to second possible implementation of second aspect, in the third possible implementation, described default
Character is 0.
The embodiment of the present invention while net cast can be generated from live terminal and is transmitted to user terminal in real time
Motion sensing control data, so that user terminal experiences the body-sensing matched with net cast content while watching video live broadcast
Simulation, drastically increase online scene can be interactive.
The third aspect, embodiments provides a kind of real-time output intent of multi-medium data, and methods described includes：
During watching video live broadcast, the picture frame that live terminal is sent in real time is received by net cast platform；
If described image frame is associated with motion sensing control packet, the live terminal is received by the net cast platform
The motion sensing control packet of synchronized transmission, the motion sensing control packet by the live terminal net cast process
It is middle to be generated according to motion sensing control instruction；
Output described image frame is to video play device, and motion sensing control packet described in synchronism output is filled to body-sensing sensing
Put.
Used as the first possible implementation of the third aspect, described in the synchronism output, motion sensing control packet is to body
Propagated sensation induction device includes：
The motion sensing control data of M body points position are parsed from the motion sensing control packet；
The body-sensing sensing device of N types is parsed in the motion sensing control data of the i-th body points position for analytically going out
Control information；
The control information for parsing is exported into the body-sensing sensing dress of the N types to i-th body points position respectively
In putting；
The M, the N and the i are the integer more than or equal to 1, and the i is less than or equal to M.
With reference to the first possible implementation of the third aspect, in second possible implementation, the synchronization
Exporting the motion sensing control packet also includes to body-sensing sensing device：
If in the motion sensing control packet, the corresponding control of body-sensing sensing device of the jth type of i-th body points position
Information processed is preset characters in each data bit, then while the related picture frame of the motion sensing control packet is played,
Stop exporting the control information；
The j is the integer more than or equal to 1, and the j is less than or equal to N.
It is with reference to second possible implementation of the third aspect, in the third possible implementation, described default
Character is 0.
Fourth aspect, embodiments provides a kind of real-time output device of multi-medium data, and described device includes：
First receiving unit, for, during watching video live broadcast, receiving live terminal by net cast platform
The picture frame for sending in real time；
Second receiving unit is if being associated with motion sensing control packet for described image frame, flat by the net cast
Platform receives the motion sensing control packet of the live terminal synchronized transmission, and the motion sensing control packet is by the live end
End is generated according to motion sensing control instruction during net cast；
Output unit, for exporting described image frame to video play device, and motion sensing control data described in synchronism output
Wrap to body-sensing sensing device.
Used as the first possible implementation of fourth aspect, the output unit includes：
First parsing subelement, for the motion sensing control of M body points position is parsed from the motion sensing control packet
Data；
Second parsing subelement, parses N species in the motion sensing control data for the i-th body points position for analytically going out
The control information of the body-sensing sensing device of type；
First output control subelement, for the control information for parsing is exported respectively to i-th body points position
On N types body-sensing sensing device in；
The M, the N and the i are the integer more than or equal to 1, and the i is less than or equal to M.
With reference to the first possible implementation of fourth aspect, in second possible implementation, the output
Unit also includes：
Second output control subelement, if in the motion sensing control packet, the jth species of i-th body points position
The corresponding control information of body-sensing sensing device of type is preset characters in each data bit, then in the motion sensing control packet
While related picture frame is played, stop exporting the control information；
The j is the integer more than or equal to 1, and the j is less than or equal to N.
It is with reference to second possible implementation of fourth aspect, in the third possible implementation, described default
Character is 0.
The embodiment of the present invention can experience the body-sensing matched with net cast content while watching video live broadcast
Simulation, drastically increase online scene can be interactive.
Description of the drawings
For the technical scheme being illustrated more clearly that in the embodiment of the present invention, below will be to embodiment or description of the prior art
Needed for accompanying drawing to be used be briefly described, it should be apparent that, drawings in the following description be only the present invention some
Embodiment, for those of ordinary skill in the art, without having to pay creative labor, can be with according to these
Accompanying drawing obtains other accompanying drawings.
Fig. 1 is the flowchart of the real-time sending method of multi-medium data provided in an embodiment of the present invention；
Fig. 2 is that real-time sending method S102 of multi-medium data provided in an embodiment of the present invention implements flow chart；
Fig. 3 is the structured flowchart of the real-time dispensing device of multi-medium data provided in an embodiment of the present invention；
Fig. 4 is a kind of schematic diagram of calculate node provided in an embodiment of the present invention；
Fig. 5 is the flowchart of the real-time output intent of multi-medium data provided in an embodiment of the present invention；
Fig. 6 is that the real-time output intent S503 of multi-medium data provided in an embodiment of the present invention implements flow chart；
Fig. 7 is the structured flowchart of the real-time output device of multi-medium data provided in an embodiment of the present invention；
Fig. 8 is the schematic diagram of another kind of calculate node provided in an embodiment of the present invention.
Specific embodiment
In below describing, in order to illustrate rather than in order to limit, it is proposed that the such as tool of particular system structure, technology etc
Body details, thoroughly to understand the embodiment of the present invention.However, it will be clear to one skilled in the art that not having these concrete
The present invention can also be realized in the other embodiments of details.In other situations, omit to well-known system, device, electricity
Road and the detailed description of method, in case unnecessary details hinders description of the invention.
The real-time Transmission scheme of the multi-medium data mentioned by the embodiment of the present invention, can during net cast,
Insert motion sensing control data in live video stream at any time, and motion sensing control data are straight by video with live video stream
Broadcast platform and be transferred to user terminal so that user terminal during watching video live broadcast can to experience body-sensing simultaneously anti-
Feedback.In embodiments of the present invention, the motion sensing control data are for being controlled to body-sensing sensing device, motion sensing control data with
The form of motion sensing control signal is sent from motion sensing control device, and is sent to the body-sensing sensing dress electrically connected with motion sensing control device
In putting.In actual product form, motion sensing control device and the body-sensing sensing device of one or more can be with wearable products
Combine, for example, motion sensing control device and multiple body-sensing sensing devices be attached to into parcel wearable of user's whole body in vivo,
Motion sensing control data are exported respectively to multiple body-sensing sensing devices by motion sensing control device, so that this multiple body-sensing sensing
Device carries out proprioceptive simulation to user according to the motion sensing control data being respectively received.
First, the real-time sending method of multi-medium data provided in an embodiment of the present invention is described in detail, its realization
Flow process is as shown in Figure 1：
In S101, during net cast, instruction monitoring is carried out to live terminal.
Here, net cast refer mainly to using the Internet and stream media technology carried out it is live, live terminal can be with
Refer to for gathering the acquisition terminal of picture signal and acoustical signal, can be the audio-video collection video camera of specialty, can also be
Smart mobile phone, flat board etc. possess the intelligent terminal of camera function.Net cast process, here are generally referred to audio frequency and video
The image and audio digital signal that acquisition terminal is collected generates live file by streaming coding device, and passes through net cast
Platform is transmitted to user terminal.
In embodiments of the present invention, when live terminal is in net cast process, instruction monitoring is carried out to live terminal,
To detect that whether user has assigned motion sensing control instruction by live terminal.
In S102, when listen to the live terminal generate motion sensing control instruct when, referred to according to the motion sensing control
Order generates motion sensing control packet, and the picture frame that live terminal described in labelling is currently collected.
During net cast, live terminal is provided for assigning the operation that motion sensing control is instructed on its display interface
Menu, user can be reached by the actions menu and assign motion sensing control instruction, carry body-sensing control in the motion sensing control instruction
Data processed, for being controlled to body-sensing sensing device.Motion sensing control instruction according to listening to generates motion sensing control packet,
And in the live video stream that live terminal is collected one picture frame of labelling, the picture frame arrived for motion sensing control instruction is monitored
When the picture frame that collected.
Further, as one embodiment of the present of invention, one or many can in each motion sensing control packet, be carried
The motion sensing control data of individual body points position, and wherein the motion sensing control data of each body points position include one or more type
Body-sensing sensing device control information.Specifically, as shown in Fig. 2 the process for generating sensing control packet is as follows：
S201, obtains the motion sensing control data of M body points position, and the motion sensing control data of each body points position include N
The control information of the body-sensing sensing device of type.
The motion sensing control data of the M body points position for getting are write the motion sensing control packet by S202.
Wherein, the M and the N are the integer more than or equal to 1.
Based on the embodiment shown in Fig. 2, when M and N are equal to 1, each motion sensing control packet is in the picture frame
During broadcasting, a kind of body-sensing sensing device controlled on a body points position produces proprioceptive simulation；When M is equal to 1 more than 1 and N, often
Individual motion sensing control packet is for when the picture frame is played, the same body-sensing sensing device controlled on multiple body points positions is same
When produce proprioceptive simulation；When M and N are all higher than 1, each motion sensing control packet is various on multiple body points positions for controlling
Body-sensing sensing device produces proprioceptive simulation simultaneously.
In embodiments of the present invention, for each body points position, according to product demand, at least can with it is integrated as listed in table 1
The body-sensing sensing device of any one or more type lifted：
Table 1
Preferably, in embodiments of the present invention, it is also proposed that a kind of frame format of motion sensing control packet, the frame format
Composition includes initial control frame and body-sensing Frame as frame head.
(1) control data of some bytes is carried in initial control frame：
Exemplarily, as shown in table 2, the control data of 4 bytes, respectively control frame are included in the initial control frame
Byte 1, control frame byte 2, control frame byte 3 and control frame byte 4, each control frame byte can be according to the need of motion sensing control
Corresponding data content is write, for example, the body points covered by motion sensing control packet can be shown with control frame byte
Bit quantity, or show whether the motion sensing control packet needs controlled device to be shielded.
Table 2
(2) the motion sensing control data of the several body-sensing sensor being sequentially written in are carried in body-sensing Frame, while also
Data bit is reserved, has needed to write the motion sensing control data of some other body-sensing sensors according to exploitation for future：
In body-sensing Frame, the corresponding motion sensing control data of every kind of body-sensing sensor may be used to indicate that the body-sensing is sensed
The related motion sensing control pattern of device, or the body-sensing functional realiey related for describing the body-sensing sensor.Exemplarily, such as table
Shown in 3, the motion sensing control data of 4 bytes of every kind of body-sensing sensor correspondence, respectively motion sensing control pattern byte 1, body-sensing work(
Can byte 1, body-sensing function byte 2 and body-sensing function byte 3.
Table 3
Table 4 shows the frame structure example of a complete motion sensing control packet, as can be seen from Table 4, the body-sensing
The frame head of control data bag is the initial control frame of 4 bytes, and 13 kinds of body-sensing sensing devices have been sequentially written in body-sensing Frame
Motion sensing control data, while also reserving including 7 kinds of functions, can write the motion sensing control data of 7 kinds of body-sensing sensing devices again, and
The motion sensing control data of every kind of body-sensing sensing device are 4 bytes.
Table 4
Exemplarily, table 5 shows that 13 kinds of body-sensings that table 1 is referred to sense preposition corresponding motion sensing control data frame structure：
Table 5
In the communication process of body-sensing sensing data, motion sensing control data are generated according to frame format mentioned above
Bag, can be on the one hand unified data form by the motion sensing control Data Integration of the body-sensing sensing device of number of different types,
Propagated sensation induction device is eliminated effectively because from the caused fragmentation of different suppliers, on the other hand, by various differences
The motion sensing control data of the body-sensing sensing device of type are transmitted in writing a packet, it is also possible to dropped to a certain extent
The appearance of packet loss phenomenon in low data transmission procedure, improves the reliability of data communication.
Additionally, on the basis of motion sensing control data packet frame form mentioned above, as an enforcement of the present invention
Example, during motion sensing control packet is generated：
If not getting the control information of the body-sensing sensing device of the jth type of i-th body points position, in the body-sensing
In control data bag, corresponding each data bit of the control information writes preset characters；
The i and the j are the integer more than or equal to 1, and the i is less than or equal to the M, the j be less than or
Equal to the N.
That is, a character is preset, if the proprioceptive simulation of a certain type is not matched to certain body points position, in motion sensing control
In packet, each data bit occupied by the body-sensing sensing device of the corresponding the type in body points position is write into this and is preset
Character.For example, the motion sensing control instruction that user assigns only indicates and needs to carry out force feedback, and need not carry out other body-sensing moulds
Intend, then in motion sensing control packet, will be all in addition to the data bit occupied by device for force feedback on all body points positions
Data bit writes the preset characters.
The frame writing scheme of above motion sensing control packet can ensure that the data length one of all motion sensing control packets
Cause, so, for each the motion sensing control packet for generating, data length all same contributes in data check process
In rapidly find packet drop.
Alternatively, the preset characters are 0.
In S103, the motion sensing control packet and described image frame are associated.
Specifically the motion sensing control packet generated in S102 and the picture frame of labelling in S102 are associated, should
Incidence relation once it is determined that then can exist always, even after net cast process terminates, by user terminal from video
When playing back whole live process on live platform, the incidence relation is present all the time, ensureing proprioceptive simulation during live and
All the time identical live video stream progress is all corresponded in replayed section.
In S104, described image frame is sent to user terminal in real time by net cast platform, at the same will with it is described
Picture frame association the motion sensing control packet by the net cast platform synchronized transmission to the user terminal so that
, while described image frame is exported to video play device, motion sensing control packet described in synchronism output is extremely for the user terminal
Body-sensing sensing device.
In embodiments of the present invention, for net cast stream and motion sensing control packet, it is real-time Transmission to regarding
Frequently live platform (server), then the user terminal of all watching video live broadcasts is broadcast to by net cast platform, therefore,
While picture frame is sent to user terminal in real time by net cast platform, the motion sensing control that also will be associated with the picture frame
Packet gives the user terminal of all watching video live broadcasts, so, user by the synchronized broadcast of same net cast platform
Terminal just can synchronously carry out proprioceptive simulation while user terminal plays net cast stream in watching video live broadcast,
It is intended to the motion sensing control for truly restoring live terminal use.
Corresponding to the real-time sending method of the multi-medium data described in foregoing embodiments, Fig. 3 shows the embodiment of the present invention
The structured flowchart of the real-time dispensing device of the multi-medium data of offer.For convenience of description, illustrate only related to the present embodiment
Part.
With reference to Fig. 3, the device includes：
Monitoring unit 31, during net cast, carries out instruction monitoring to live terminal；
Signal generating unit 32, when listen to the live terminal generate motion sensing control instruct when, according to the motion sensing control
Instruction generates motion sensing control packet, and the picture frame that live terminal described in labelling is currently collected；
Associative cell 33, the motion sensing control packet and described image frame are associated；
Transmitting element 34, described image frame is sent to user terminal in real time by net cast platform, while will be with institute
The motion sensing control packet of picture frame association is stated by the net cast platform synchronized transmission to the user terminal, with
Make the user terminal export described image frame to video play device while, motion sensing control packet described in synchronism output
To body-sensing sensing device.
Alternatively, the signal generating unit 31 includes：
Subelement is obtained, the motion sensing control data of M body points position are obtained, the motion sensing control data of each body points position are equal
The control information of the body-sensing sensing device comprising N types；
The motion sensing control data of the M body points position for getting are write the body-sensing by the first write subelement
Control data bag；
The M and the N are the integer more than or equal to 1.
Alternatively, the signal generating unit 31 also includes；
Second write subelement, if not getting the control of the body-sensing sensing device of the jth type of i-th body points position
Information, in the motion sensing control packet, corresponding each data bit of the control information writes preset characters；
The i and the j are the integer more than or equal to 1, and the i is less than or equal to the M, the j be less than or
Equal to the N.
Alternatively, the preset characters are 0.
Corresponding to the real-time sending method of the multi-medium data described in foregoing embodiments, Fig. 4 shows the embodiment of the present invention
A kind of schematic diagram of the calculate node 400 for providing.For convenience of description, illustrate only part related to the present embodiment.
Wherein, the possibly host server comprising computing capability of calculate node 400, or personal computer PC, or
Person is portable portable computer or terminal etc., and the specific embodiment of the invention is not done to implementing for calculate node
Limit.Calculate node 400 includes：
Processor (processor) 410, communication interface (Communications Interface) 420, memorizer
(memory) 430, bus 440.
Processor 410, communication interface 420, memorizer 430 complete mutual communication by bus 440.
Communication interface 420, for net element communication, such as Virtual Machine Manager center, shared storage etc..
Processor 410, for configuration processor.
Specifically, program can include program code, and described program code includes computer-managed instruction.
Processor 410 is probably a central processing unit (Central Processing Unit, CPU), or specific
Integrated circuit (Application Specific Integrated Circuit, ASIC), or be arranged to implement this
One or more integrated circuits of bright embodiment.
Memorizer 430, for depositing program.Memorizer 430 may include high-speed RAM memorizer, it is also possible to also including non-
Volatile memory (non-volatile memory), for example, at least one disk memory.
Program specifically can be used for the real-time sending method for performing a kind of multi-medium data, and methods described includes：
During net cast, instruction monitoring is carried out to live terminal；
When listen to the live terminal generate motion sensing control instruct when, according to the motion sensing control instruction generate body-sensing
Control data bag, and the picture frame that live terminal described in labelling is currently collected；
The motion sensing control packet and described image frame are associated；
Described image frame is sent to user terminal in real time by net cast platform, while will associate with described image frame
The motion sensing control packet by the net cast platform synchronized transmission to the user terminal so that user's end
While described image frame is exported to video play device, motion sensing control packet described in synchronism output is filled to body-sensing sensing at end
Put.
Alternatively, it is described to be included according to motion sensing control instruction generation motion sensing control packet：
The motion sensing control data of M body points position are obtained, the motion sensing control data of each body points position include N types
Body-sensing sensing device control information；
The motion sensing control data of the M body points position for getting are write into the motion sensing control packet；
The M and the N are the integer more than or equal to 1.
Alternatively, it is described also to be included according to motion sensing control instruction generation motion sensing control packet；
If not getting the control information of the body-sensing sensing device of the jth type of i-th body points position, in the body-sensing
In control data bag, corresponding each data bit of the control information writes preset characters；
The i and the j are the integer more than or equal to 1, and the i is less than or equal to the M, the j be less than or
Equal to the N.
Alternatively, the preset characters are 0.
The embodiment of the present invention while net cast can be generated from live terminal and is transmitted to user terminal in real time
Motion sensing control data, so that user terminal experiences the body-sensing matched with net cast content while watching video live broadcast
Simulation, drastically increase online scene can be interactive.
Next, the real-time output intent of multi-medium data provided in an embodiment of the present invention is described in detail, in fact
Existing flow process is as shown in Figure 5：
In S501, during watching video live broadcast, live terminal is received by net cast platform and is sent in real time
Picture frame.
In S502, if described image frame is associated with motion sensing control packet, institute is received by the net cast platform
The motion sensing control packet of live terminal synchronized transmission is stated, the motion sensing control packet is by the live terminal in video
Generated according to motion sensing control instruction in live process.
In S503, output described image frame is to video play device, and motion sensing control packet described in synchronism output is extremely
Body-sensing sensing device.
In embodiments of the present invention, user terminal is by the way of watching video live broadcast in prior art, from net cast
Platform receives the picture frame that live terminal is sent in real time, and as described in embodiment above, if live terminal use is sometime
Motion sensing control instruction is assigned, then the motion sensing control packet generated according to the sensing control instruction can have been collected with the moment
Picture frame by net cast platform synchronized transmission to user terminal, then user terminal export the picture frame broadcast to video
While putting device, also the synchronism output motion sensing control packet is to body-sensing sensing device.The life of above-mentioned motion sensing control packet
Elaborated in foregoing embodiments into method, data form and data content, will not be described here.
As one embodiment of the present of invention, when the N kinds on M body points position are contained in each motion sensing control packet
During the control information of the body-sensing sensing device of type, then as shown in fig. 6, motion sensing control packet is to body described in the synchronism output
Propagated sensation induction device includes：
S601, parses the motion sensing control data of M body points position from the motion sensing control packet.
S602, parses the body-sensing sensing dress of N types in the motion sensing control data of the i-th body points position for analytically going out
The control information put.
S603, the body-sensing that the control information for parsing exports the N types to i-th body points position respectively is passed
In induction device.
The M, the N and the i are the integer more than or equal to 1, and the i is less than or equal to M.
Based on the corresponding embodiments of Fig. 6, during motion sensing control device output motion sensing control packet, can be accurate
Ground is by motion sensing control data output in a concrete body-sensing sensing device of a concrete body points position, then for whole body M
M*N individualities propagated sensation induction device altogether on individual body points position, just can realize body during related picture frame is played simultaneously
The output of sense simulation, brings omnibearing body-sensing experience.
Additionally, as described above, a character is preset on the basis of above-mentioned motion sensing control data packet frame form, if
The proprioceptive simulation of a certain type is not matched to certain body points position, then in the motion sensing control packet, will be the body points position right
Each data bit occupied by the body-sensing sensing device of the type answered writes the preset characters, for example, writes preset characters
0, it is consistent with the data length for ensureing all motion sensing control packets.So under the program, body described in the synchronism output
Sensing control packet also includes to body-sensing sensing device：
If in the motion sensing control packet, the corresponding control of body-sensing sensing device of the jth type of i-th body points position
Information processed is preset characters in each data bit, then while the related picture frame of the motion sensing control packet is played,
Stop exporting the control information；
The j is the integer more than or equal to 1, and the j is less than or equal to N.
That is, if concrete certain class body-sensing sensing device that motion sensing control device is detected on concrete certain body points position is corresponding
Preset characters are in each occupied in body-sensing packet data bit of control information, are then stopped to the body at the moment
The body-sensing sensing device output control information on point position, so, although the related motion sensing control data of each picture frame
The data length of bag is identical, but motion sensing control device still can be judged according to the data content in each data bit
It is that output control information is also off output control information, so as to neatly realize to many body points positions, multiple body-sensings sensing
The output control of device.
The embodiment of the present invention while net cast can be generated from live terminal and is transmitted to user terminal in real time
Motion sensing control data, so, user terminal can be experienced while watching video live broadcast and net cast content phase
The proprioceptive simulation of matching, drastically increase online scene can be interactive.
Corresponding to the real-time output intent of the multi-medium data described in foregoing embodiments, Fig. 7 shows the embodiment of the present invention
The structured flowchart of the real-time output device of the multi-medium data of offer.For convenience of description, illustrate only related to the present embodiment
Part.
With reference to Fig. 7, the device includes：
First receiving unit 71, during watching video live broadcast, receives live terminal reality by net cast platform
When the picture frame that sends；
Second receiving unit 72, if described image frame is associated with motion sensing control packet, by the net cast platform
The motion sensing control packet of the live terminal synchronized transmission is received, the motion sensing control packet is by the live terminal
Generated according to motion sensing control instruction during net cast；
Output unit 73, output described image frame is to video play device, and motion sensing control packet described in synchronism output
To body-sensing sensing device.
Alternatively, the output unit 73 includes：
First parsing subelement, parses the motion sensing control data of M body points position from the motion sensing control packet；
Second parsing subelement, parses N types in the motion sensing control data of the i-th body points position for analytically going out
The control information of body-sensing sensing device；
The control information for parsing is exported the N to i-th body points position by the first output control subelement respectively
In the body-sensing sensing device of type；
The M, the N and the i are the integer more than or equal to 1, and the i is less than or equal to M.
Alternatively, the output unit 73 also includes：
Second output control subelement, if in the motion sensing control packet, the jth type of i-th body points position
The corresponding control information of body-sensing sensing device is preset characters in each data bit, then related in the motion sensing control packet
Picture frame play while, stop export the control information；
The j is the integer more than or equal to 1, and the j is less than or equal to N.
Alternatively, the preset characters are 0.
Corresponding to the real-time output intent of the multi-medium data described in foregoing embodiments, Fig. 8 shows the embodiment of the present invention
A kind of schematic diagram of the calculate node 800 for providing.For convenience of description, illustrate only part related to the present embodiment.
Wherein, the possibly host server comprising computing capability of calculate node 800, or personal computer PC, or
Person is portable portable computer or terminal etc., and the specific embodiment of the invention is not done to implementing for calculate node
Limit.Calculate node 800 includes：
Processor (processor) 810, communication interface (Communications Interface) 820, memorizer
(memory) 830, bus 840.
Processor 810, communication interface 820, memorizer 830 complete mutual communication by bus 840.
Communication interface 820, for net element communication, such as Virtual Machine Manager center, shared storage etc..
Processor 810, for configuration processor.
Specifically, program can include program code, and described program code includes computer-managed instruction.
Processor 810 is probably a central processing unit (Central Processing Unit, CPU), or specific
Integrated circuit (Application Specific Integrated Circuit, ASIC), or be arranged to implement this
One or more integrated circuits of bright embodiment.
Memorizer 830, for depositing program.Memorizer 830 may include high-speed RAM memorizer, it is also possible to also including non-
Volatile memory (non-volatile memory), for example, at least one disk memory.
Program specifically can be used for the real-time output intent for performing a kind of multi-medium data, and methods described includes：
During watching video live broadcast, the picture frame that live terminal is sent in real time is received by net cast platform；
If described image frame is associated with motion sensing control packet, the live terminal is received by the net cast platform
The motion sensing control packet of synchronized transmission, the motion sensing control packet by the live terminal net cast process
It is middle to be generated according to motion sensing control instruction；
Output described image frame is to video play device, and motion sensing control packet described in synchronism output is filled to body-sensing sensing
Put.
Alternatively, described in the synchronism output, motion sensing control packet includes to body-sensing sensing device：
The motion sensing control data of M body points position are parsed from the motion sensing control packet；
The body-sensing sensing device of N types is parsed in the motion sensing control data of the i-th body points position for analytically going out
Control information；
The control information for parsing is exported into the body-sensing sensing dress of the N types to i-th body points position respectively
In putting；
The M, the N and the i are the integer more than or equal to 1, and the i is less than or equal to M.
Alternatively, described in the synchronism output, motion sensing control packet also includes to body-sensing sensing device：
If in the motion sensing control packet, the corresponding control of body-sensing sensing device of the jth type of i-th body points position
Information processed is preset characters in each data bit, then while the related picture frame of the motion sensing control packet is played,
Stop exporting the control information；
The j is the integer more than or equal to 1, and the j is less than or equal to N.
Alternatively, the preset characters are 0.
Those skilled in the art can be understood that, for convenience of description and succinctly, only with above-mentioned each work(
The division of energy unit is illustrated, and in practical application, can distribute above-mentioned functions by different functions as desired
Unit is completed, will the internal structure of device be divided into different functional units or module, with complete it is described above whole or
Person's partial function.During each functional unit in embodiment can be integrated in a processing unit, or unit is independent
It is physically present, it is also possible to which two or more units are integrated in a unit, above-mentioned integrated unit both can be using hard
The form of part is realized, it would however also be possible to employ the form of SFU software functional unit is realized.In addition, the specific name of each functional unit is also
For the ease of mutually differentiation, the protection domain of the application is not limited to.The specific work process of unit in said apparatus, can
With reference to the corresponding process in aforementioned means embodiment, will not be described here.
Those of ordinary skill in the art are it is to be appreciated that the list of each example described with reference to the embodiments described herein
Unit and algorithm steps, being capable of being implemented in combination in electronic hardware or computer software and electronic hardware.These functions are actually
Performed with hardware or software mode, depending on the application-specific and design constraint of technical scheme.Professional and technical personnel
Described function, but this realization can be realized it is not considered that exceeding to each specific application using different device
The scope of the present invention.
In embodiment provided by the present invention, it should be understood that disclosed device and device, can pass through other
Mode is realized.For example, device embodiment described above is only schematically, for example division of module or unit, only
For a kind of division of logic function, when actually realizing, there can be other dividing mode, such as multiple units or component can be combined
Or another system is desirably integrated into, or some features can be ignored, or do not perform.Another, shown or discussed phase
Coupling or direct-coupling or communication connection between mutually can be INDIRECT COUPLING or the communication by some interfaces, device or unit
Connection, can be electrical, mechanical or other forms.
As the unit of separating component explanation can be or may not be physically separate, show as unit
Part can be or may not be physical location, you can local to be located at one, or can also be distributed to multiple networks
On unit.Some or all of unit therein can be selected according to the actual needs to realize the purpose of this embodiment scheme.
In addition, each functional unit in each embodiment of the invention can be integrated in a processing unit, it is also possible to
It is that unit is individually physically present, it is also possible to which two or more units are integrated in a unit.Above-mentioned integrated list
Unit both can be realized in the form of hardware, it would however also be possible to employ the form of SFU software functional unit is realized.
If integrated unit realized using in the form of SFU software functional unit and as independent production marketing or use when, can
To be stored in a computer read/write memory medium.Based on such understanding, the technical scheme essence of the embodiment of the present invention
On all or part of part that in other words prior art is contributed or the technical scheme can be with the shape of software product
Formula is embodied, and the computer software product is stored in a storage medium, is used including some instructions so that a calculating
Machine equipment (can be personal computer, server, or network equipment etc.) or the computing device embodiment of the present invention each reality
Apply all or part of step of a device.And aforesaid storage medium includes：USB flash disk, portable hard drive, read only memory, deposit at random
Access to memory, magnetic disc or CD etc. are various can be with the medium of store program codes.
Above example only to illustrate technical scheme, rather than a limitation；Although with reference to the foregoing embodiments
The present invention has been described in detail, it will be understood by those within the art that：Which still can be to aforementioned each enforcement
Technical scheme described in example is modified, or carries out equivalent to which part technical characteristic；And these modification or
Replace, do not make the essence of appropriate technical solution depart from the spirit and scope of each embodiment technical scheme of the embodiment of the present invention.
These are only presently preferred embodiments of the present invention, not to limit the present invention, all spirit in the present invention and
Any modification, equivalent and improvement for being made within principle etc., should be included within the scope of the present invention.
Claims (16)
1. the real-time sending method of a kind of multi-medium data, it is characterised in that methods described includes：
During net cast, instruction monitoring is carried out to live terminal；
When listen to the live terminal generate motion sensing control instruct when, according to the motion sensing control instruction generate motion sensing control
Packet, and the picture frame that live terminal described in labelling is currently collected；
The motion sensing control packet and described image frame are associated；
Described image frame is sent to user terminal in real time by net cast platform, while the institute that will be associated with described image frame
Motion sensing control packet is stated by the net cast platform synchronized transmission to the user terminal, so that the user terminal exists
While output described image frame is to video play device, described in synchronism output, motion sensing control packet is to body-sensing sensing device.
2. the method for claim 1, it is characterised in that described that motion sensing control number is generated according to motion sensing control instruction
Include according to bag：
Obtain the motion sensing control data of M body points position, the motion sensing control data body comprising N types of each body points position
The control information of propagated sensation induction device；
The motion sensing control data of the M body points position for getting are write into the motion sensing control packet；
The M and the N are the integer more than or equal to 1.
3. method as claimed in claim 2, it is characterised in that described that motion sensing control number is generated according to motion sensing control instruction
Also include according to bag；
If not getting the control information of the body-sensing sensing device of the jth type of i-th body points position, in the motion sensing control
In packet, corresponding each data bit of the control information writes preset characters；
The i and the j are the integer more than or equal to 1, and the i is less than or equal to the M, and the j is less than or equal to
The N.
4. method as claimed in claim 3, it is characterised in that the preset characters are 0.
5. the real-time output intent of a kind of multi-medium data, it is characterised in that methods described includes：
During watching video live broadcast, the picture frame that live terminal is sent in real time is received by net cast platform；
If described image frame is associated with motion sensing control packet, the live terminal synchronization is received by the net cast platform
The motion sensing control packet for sending, the motion sensing control packet is by live terminal root during net cast
Generate according to motion sensing control instruction；
Output described image frame is to video play device, and motion sensing control packet described in synchronism output is to body-sensing sensing device.
6. method as claimed in claim 5, it is characterised in that motion sensing control packet described in the synchronism output to body-sensing is passed
Induction device includes：
The motion sensing control data of M body points position are parsed from the motion sensing control packet；
The control of the body-sensing sensing device of N types is parsed in the motion sensing control data of the i-th body points position for analytically going out
Information；
The control information for parsing is exported in the body-sensing sensing device of the N types to i-th body points position respectively；
The M, the N and the i are the integer more than or equal to 1, and the i is less than or equal to M.
7. method as claimed in claim 6, it is characterised in that motion sensing control packet described in the synchronism output to body-sensing is passed
Induction device also includes：
If in the motion sensing control packet, the corresponding control letter of body-sensing sensing device of the jth type of i-th body points position
Breath is preset characters in each data bit, then, while the related picture frame of the motion sensing control packet is played, stop
Export the control information；
The j is the integer more than or equal to 1, and the j is less than or equal to N.
8. method as claimed in claim 7, it is characterised in that the preset characters are 0.
9. the real-time dispensing device of a kind of multi-medium data, it is characterised in that described device includes：
Monitoring unit, for, during net cast, carrying out instruction monitoring to live terminal；
Signal generating unit, for when listen to the live terminal generate motion sensing control instruct when, referred to according to the motion sensing control
Order generates motion sensing control packet, and the picture frame that live terminal described in labelling is currently collected；
Associative cell, for the motion sensing control packet and described image frame are associated；
Transmitting element, for described image frame is sent to user terminal in real time by net cast platform, while will with it is described
Picture frame association the motion sensing control packet by the net cast platform synchronized transmission to the user terminal so that
, while described image frame is exported to video play device, motion sensing control packet described in synchronism output is extremely for the user terminal
Body-sensing sensing device.
10. device as claimed in claim 9, it is characterised in that the signal generating unit includes：
Subelement is obtained, for obtaining the motion sensing control data of M body points position, the motion sensing control data of each body points position are equal
The control information of the body-sensing sensing device comprising N types；
First write subelement, for the motion sensing control data of the M body points position for getting are write the body-sensing
Control data bag；
The M and the N are the integer more than or equal to 1.
11. devices as claimed in claim 10, it is characterised in that the signal generating unit also includes；
Second write subelement, if the control of the body-sensing sensing device of the jth type for not getting i-th body points position
Information, in the motion sensing control packet, corresponding each data bit of the control information writes preset characters；
The i and the j are the integer more than or equal to 1, and the i is less than or equal to the M, and the j is less than or equal to
The N.
12. devices as claimed in claim 11, it is characterised in that the preset characters are 0.
The real-time output device of 13. a kind of multi-medium datas, it is characterised in that described device includes：
First receiving unit, for, during watching video live broadcast, receiving live terminal by net cast platform real-time
The picture frame of transmission；
Second receiving unit, if being associated with motion sensing control packet for described image frame, is connect by the net cast platform
The motion sensing control packet of the live terminal synchronized transmission is received, the motion sensing control packet is existed by the live terminal
Generated according to motion sensing control instruction during net cast；
Output unit, for exporting described image frame to video play device, and motion sensing control packet described in synchronism output is extremely
Body-sensing sensing device.
14. devices as claimed in claim 13, it is characterised in that the output unit includes：
First parsing subelement, for the motion sensing control data of M body points position are parsed from the motion sensing control packet；
Second parsing subelement, parses N types in the motion sensing control data for the i-th body points position for analytically going out
The control information of body-sensing sensing device；
First output control subelement, for the control information for parsing to be exported N to i-th body points position respectively
In the body-sensing sensing device of type；
The M, the N and the i are the integer more than or equal to 1, and the i is less than or equal to M.
15. devices as claimed in claim 14, it is characterised in that the output unit also includes：
Second output control subelement, if in the motion sensing control packet, the jth type of i-th body points position
The corresponding control information of body-sensing sensing device is preset characters in each data bit, then related in the motion sensing control packet
Picture frame play while, stop export the control information；
The j is the integer more than or equal to 1, and the j is less than or equal to N.
16. devices as claimed in claim 14, it is characterised in that the preset characters are 0.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611058946.9A CN106534142B (en) | 2016-11-22 | 2016-11-22 | The live transmission method and device of multi-medium data |
| PCT/CN2017/086495 WO2018095003A1 (en) | 2016-11-22 | 2017-05-29 | Method and apparatus for real-time transmission of multimedia data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611058946.9A CN106534142B (en) | 2016-11-22 | 2016-11-22 | The live transmission method and device of multi-medium data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106534142A true CN106534142A (en) | 2017-03-22 |
| CN106534142B CN106534142B (en) | 2018-04-20 |
Family
ID=58357196
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611058946.9A Active CN106534142B (en) | 2016-11-22 | 2016-11-22 | The live transmission method and device of multi-medium data |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN106534142B (en) |
| WO (1) | WO2018095003A1 (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107491172A (en) * | 2017-08-16 | 2017-12-19 | 歌尔科技有限公司 | Body-sensing data capture method, device and electronic equipment |
| WO2018095003A1 (en) * | 2016-11-22 | 2018-05-31 | 包磊 | Method and apparatus for real-time transmission of multimedia data |
| CN109840016A (en) * | 2018-12-14 | 2019-06-04 | 维沃移动通信有限公司 | A kind of virtual interacting method and terminal device |
| WO2019127369A1 (en) * | 2017-12-29 | 2019-07-04 | 腾讯科技（深圳）有限公司 | Live broadcast sharing method, and related device and system |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101828382A (en) * | 2007-10-16 | 2010-09-08 | 伊梅森公司 | Haptic effect data in the media transport stream synchronously |
| CN102598085A (en) * | 2009-08-14 | 2012-07-18 | K12股份有限公司 | Systems and methods for producing, delivering and managing educational material |
| US20130073673A1 (en) * | 2011-09-19 | 2013-03-21 | Comcast Cable Communications, LLC. | Content Storage and Identification |
| CN103970266A (en) * | 2013-01-24 | 2014-08-06 | 英默森公司 | Haptic sensation recording and playback |
| CN104049749A (en) * | 2013-03-15 | 2014-09-17 | 英默森公司 | Method and apparatus to generate haptic feedback from video content analysis |
| CN104932678A (en) * | 2014-03-21 | 2015-09-23 | 意美森公司 | Systems and methods for converting sensory data to haptic effects |
| CN105721814A (en) * | 2014-12-19 | 2016-06-29 | 意美森公司 | Systems and methods for recording haptic data for use with multi-media data |
| US20160366450A1 (en) * | 2015-06-12 | 2016-12-15 | Immersion Corporation | Broadcast haptics architectures |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080263620A1 (en) * | 2005-12-23 | 2008-10-23 | Koninklijke Philips Electronics N. V. | Script Synchronization Using Fingerprints Determined From a Content Stream |
| KR101131856B1 (en) * | 2006-11-03 | 2012-03-30 | 엘지전자 주식회사 | Apparatus For Transmitting Broadcast Signal And Method Of Transmitting And Receiving Broadcast Signal Using Same |
| US20100275235A1 (en) * | 2007-10-16 | 2010-10-28 | Sanghyun Joo | Sensory effect media generating and consuming method and apparatus thereof |
| JP2012524452A (en) * | 2009-04-15 | 2012-10-11 | エレクトロニクス アンド テレコミュニケーションズ リサーチ インスチチュート | Method and apparatus for providing metadata for sensory effect, computer-readable recording medium on which metadata for sensory effect is recorded, sensory reproduction method and apparatus |
| US20150054727A1 (en) * | 2013-08-23 | 2015-02-26 | Immersion Corporation | Haptically enabled viewing of sporting events |
| CN104780403A (en) * | 2014-03-28 | 2015-07-15 | 方小祥 | System and method for providing somatosensory video based on script |
| US9667907B2 (en) * | 2014-09-13 | 2017-05-30 | Vicente Diaz | System and method for haptic integration and generation in multimedia devices |
| CN106534142B (en) * | 2016-11-22 | 2018-04-20 | 包磊 | The live transmission method and device of multi-medium data |
-
2016
- 2016-11-22 CN CN201611058946.9A patent/CN106534142B/en active Active
-
2017
- 2017-05-29 WO PCT/CN2017/086495 patent/WO2018095003A1/en active Application Filing
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101828382A (en) * | 2007-10-16 | 2010-09-08 | 伊梅森公司 | Haptic effect data in the media transport stream synchronously |
| CN102598085A (en) * | 2009-08-14 | 2012-07-18 | K12股份有限公司 | Systems and methods for producing, delivering and managing educational material |
| US20130073673A1 (en) * | 2011-09-19 | 2013-03-21 | Comcast Cable Communications, LLC. | Content Storage and Identification |
| CN103970266A (en) * | 2013-01-24 | 2014-08-06 | 英默森公司 | Haptic sensation recording and playback |
| CN104049749A (en) * | 2013-03-15 | 2014-09-17 | 英默森公司 | Method and apparatus to generate haptic feedback from video content analysis |
| CN104932678A (en) * | 2014-03-21 | 2015-09-23 | 意美森公司 | Systems and methods for converting sensory data to haptic effects |
| CN105721814A (en) * | 2014-12-19 | 2016-06-29 | 意美森公司 | Systems and methods for recording haptic data for use with multi-media data |
| US20160366450A1 (en) * | 2015-06-12 | 2016-12-15 | Immersion Corporation | Broadcast haptics architectures |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018095003A1 (en) * | 2016-11-22 | 2018-05-31 | 包磊 | Method and apparatus for real-time transmission of multimedia data |
| CN107491172A (en) * | 2017-08-16 | 2017-12-19 | 歌尔科技有限公司 | Body-sensing data capture method, device and electronic equipment |
| WO2019127369A1 (en) * | 2017-12-29 | 2019-07-04 | 腾讯科技（深圳）有限公司 | Live broadcast sharing method, and related device and system |
| CN109840016A (en) * | 2018-12-14 | 2019-06-04 | 维沃移动通信有限公司 | A kind of virtual interacting method and terminal device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106534142B (en) | 2018-04-20 |
| WO2018095003A1 (en) | 2018-05-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104618797B (en) | Information processing method, device and client | |
| CN106534142B (en) | The live transmission method and device of multi-medium data | |
| CN104574005B (en) | Collect augmented reality, body-sensing, the advertising display management system and method for scratching green technology | |
| CN107835971A (en) | Based on user's tactual space（HapSpace）The method and apparatus that touch feedback and interaction are provided | |
| CN104796448B (en) | The data processing method and device of network system | |
| CN101174332B (en) | Method, device and system for interactively combining real-time scene in real world with virtual reality scene | |
| CN105227550B (en) | Scene display methods, apparatus and system | |
| CN106095235A (en) | Control method based on virtual reality and device | |
| CN113240782A (en) | Streaming media generation method and device based on virtual role | |
| CN102349063B (en) | Tear-free remote desktop protocol (RDP) display | |
| CN106249868A (en) | Broadcast sense of touch framework | |
| CN108134964A (en) | Net cast stage property stacking method, computer equipment and storage medium | |
| CN108632676A (en) | Display methods, device, storage medium and the electronic device of image | |
| CN109600628A (en) | Video creating method, device, computer equipment and storage medium | |
| CN103324488B (en) | A kind of special-effect information acquisition methods and device | |
| CN106527730B (en) | Generation, output control method and the device of motion sensing control data | |
| CN107948702A (en) | Synchronous method, device, terminal and the storage medium of Application Status | |
| CN108983974B (en) | AR scene processing method, device, equipment and computer-readable storage medium | |
| CN114510150A (en) | Experience system of virtual digital world | |
| CN106980382A (en) | A kind of method, mobile terminal and the VR equipment of the control of VR device plays | |
| Kim et al. | Construction of a haptic-enabled broadcasting system based on the MPEG-V standard | |
| CN107172136A (en) | The synchronous method and device of voxel data | |
| CN106454365B (en) | Coding, coding/decoding method and the coding of multi-medium data, decoding apparatus | |
| CN106547357B (en) | The communication processing method and device of body-sensing sensing data | |
| CN108733216A (en) | A kind of enterprise rollout transaction platform based on three-dimensional exclusive business card |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |