CN106534142B - The live transmission method and device of multi-medium data - Google Patents
The live transmission method and device of multi-medium data Download PDFInfo
- Publication number
- CN106534142B CN106534142B CN201611058946.9A CN201611058946A CN106534142B CN 106534142 B CN106534142 B CN 106534142B CN 201611058946 A CN201611058946 A CN 201611058946A CN 106534142 B CN106534142 B CN 106534142B
- Authority
- CN
- China
- Prior art keywords
- motion sensing
- control data
- sensing control
- data packet
- sensing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 49
- 230000005540 biological transmission Effects 0.000 title claims description 8
- 238000012544 monitoring process Methods 0.000 claims abstract description 12
- 230000001360 synchronised effect Effects 0.000 claims abstract description 3
- 230000006698 induction Effects 0.000 claims description 9
- 230000000644 propagated effect Effects 0.000 claims description 6
- 230000035807 sensation Effects 0.000 claims description 6
- 238000004088 simulation Methods 0.000 abstract description 14
- 230000000272 proprioceptive effect Effects 0.000 abstract description 10
- 230000002452 interceptive effect Effects 0.000 abstract description 9
- 230000006854 communication Effects 0.000 description 15
- 238000004891 communication Methods 0.000 description 14
- 230000006870 function Effects 0.000 description 12
- 230000008569 process Effects 0.000 description 12
- 238000010586 diagram Methods 0.000 description 8
- 238000003860 storage Methods 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 6
- 238000012545 processing Methods 0.000 description 6
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000008878 coupling Effects 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 241000894007 species Species 0.000 description 2
- 230000006399 behavior Effects 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 230000005611 electricity Effects 0.000 description 1
- 238000013467 fragmentation Methods 0.000 description 1
- 238000006062 fragmentation reaction Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000003238 somatosensory effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
- 238000010792 warming Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/24—Monitoring of processes or resources, e.g. monitoring of server load, available bandwidth, upstream requests
- H04N21/2408—Monitoring of the upstream path of the transmission network, e.g. client requests
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
- H04N21/258—Client or end-user data management, e.g. managing client capabilities, user preferences or demographics, processing of multiple end-users preferences to derive collaborative data
- H04N21/25866—Management of end-user data
- H04N21/25891—Management of end-user data being end-user preferences
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Computer Graphics (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
- Selective Calling Equipment (AREA)
Abstract
The present invention is suitable for network technique field, there is provided during net cast, instruction monitoring is carried out to live terminal；When listening to the live terminal and generating motion sensing control instruction, generation motion sensing control data packet, and the picture frame for marking the live terminal currently to collect are instructed according to the motion sensing control；The motion sensing control data packet is associated with described image frame；Described image frame is sent to the user terminal in real time by net cast platform, it will be sent at the same time with the associated motion sensing control data packet of described image frame by the way that the net cast platform is synchronous to the user terminal, so that the user terminal is while described image frame to video play device is exported, motion sensing control data packet described in synchronism output to body-sensing sensing device.The present invention makes user terminal experience the proprioceptive simulation to match with net cast content while watching video live broadcast, and drastically increase online scene can be interactive.
Description
Technical field
The invention belongs to the live transmission method and device of network technique field, more particularly to multi-medium data.
Background technology
Multimedia refers to a kind of man-machine interactive information interchange and the communications media for combining two or more media, makes
Media include word, picture, sound, animation and film, and the interaction function that formula is provided.With science and technology
Development, multimedia technology can also have people except that can bring visual experience and audio experience by computer for user
Other body-sensings having are simulated, including to sense possessed by all human bodies such as sense of touch, power sense, warming, wet sense, wind sense, smell
Know and simulated, to bring the interactive experience in terms of body-sensing for user.
Body-sensing sensing realization principle be：Motion sensing control signal is sent to the body-sensing sensing device for being attached to body surface,
All kinds of body-sensing feedbacks are carried out to human body with control volume propagated sensation induction device.For example, at virtual reality (Virtual Reality, VR)
When shaking hands in game with virtual portrait, by force-feedback control signal be pointed to hand force snesor produce strength and
Duration is controlled, so as to bring the force feedback to match with the behavior of shaking hands in game to experience.However, technical staff exists
Found in R＆D process, at least there are following technological deficiency for the prior art：
It is greatly that user oriented is played online in the streaming media that existing video file, which has, and existing
Video content provider and all kinds of somatosensory device suppliers each separately provide service, video content is believed with body-sensing sensing control
Number form it is also different, therefore, the prior art can not be the video real-time matching body-sensing data that play online, thus reduce
Online scene it is interactive.
The content of the invention
In view of this, it is existing to solve an embodiment of the present invention provides the live transmission method and device of multi-medium data
Technology can not be the video real-time matching body-sensing data that play online, thus reduce online scene it is interactive the problem of.
In a first aspect, an embodiment of the present invention provides a kind of real-time sending method of multi-medium data, the described method includes：
During net cast, instruction monitoring is carried out to live terminal；
When listen to the live terminal generate motion sensing control instruction when, according to the motion sensing control instruct generation body-sensing
Control data packet, and the picture frame for marking the live terminal currently to collect；
The motion sensing control data packet is associated with described image frame；
Described image frame is sent to the user terminal in real time by net cast platform, while will be associated with described image frame
The motion sensing control data packet by the net cast platform synchronously send to the user terminal so that the user is whole
While described image frame to video play device is exported, motion sensing control data packet described in synchronism output to body-sensing, which senses, to be filled at end
Put.
It is described that generation body-sensing control is instructed according to the motion sensing control as the first possible implementation of first aspect
Data packet processed includes：
The motion sensing control data of M body points position are obtained, the motion sensing control data of each body points position include N types
Body-sensing sensing device control information；
The motion sensing control data of the M body points position got are write into the motion sensing control data packet；
The M and N is the integer more than or equal to 1.
The possible implementation of with reference to first aspect the first, in second of possible implementation, the basis
The motion sensing control instruction generation motion sensing control data packet further includes；
If the control information of the body-sensing sensing device of the jth type of i-th of body points position is not got, in the body-sensing
The corresponding each data bit of the control information writes preset characters in control data packet；
The i and j is the integer more than or equal to 1, and the i is less than or equal to the M, the j be less than or
Equal to the N.
Second of possible implementation with reference to first aspect, it is described default in the third possible implementation
Character is 0.
Second aspect, a kind of real-time dispensing device of multi-medium data, described device include：
Monitoring unit, for during net cast, instruction monitoring to be carried out to live terminal；
Generation unit, for when listen to the live terminal generate motion sensing control instruction when, according to the body-sensing control
System instruction generation motion sensing control data packet, and the picture frame for marking the live terminal currently to collect；
Associative cell, for the motion sensing control data packet to be associated with described image frame；
Transmitting element, for described image frame to be sent to the user terminal in real time by net cast platform, while will be with
The associated motion sensing control data packet of described image frame is synchronously sent to the user terminal by the net cast platform,
So that the user terminal is while described image frame to video play device is exported, motion sensing control data described in synchronism output
Wrap to body-sensing sensing device.
As the first possible implementation of second aspect, the generation unit includes：
Subelement is obtained, for obtaining the motion sensing control data of M body points position, the motion sensing control number of each body points position
According to the control information for the body-sensing sensing device for including N types；
First write-in subelement, it is described for the motion sensing control data of the M body points position got to be write
Motion sensing control data packet；
The M and N is the integer more than or equal to 1.
With reference to the first possible implementation of second aspect, in second of possible implementation, the generation
Unit further includes；
Second writes subelement, if the body-sensing sensing device of the jth type for not getting i-th of body points position
Control information, the corresponding each data bit of the control information writes preset characters in the motion sensing control data packet；
The i and j is the integer more than or equal to 1, and the i is less than or equal to the M, the j be less than or
Equal to the N.
It is described default in the third possible implementation with reference to second of possible implementation of second aspect
Character is 0.
The embodiment of the present invention can be generated and transmitted to user terminal while net cast from live terminal in real time
Motion sensing control data, so that user terminal experiences the body-sensing to match with net cast content while watching video live broadcast
Simulation, drastically increase online scene can be interactive.
The third aspect, an embodiment of the present invention provides a kind of real-time output method of multi-medium data, the described method includes：
During watching video live broadcast, the picture frame that live terminal sends in real time is received by net cast platform；
If described image frame is associated with motion sensing control data packet, the live terminal is received by the net cast platform
The motion sensing control data packet synchronously sent, the motion sensing control data packet by the live terminal net cast process
Middle instructed according to motion sensing control generates；
Described image frame is exported to video play device, and motion sensing control data packet described in synchronism output is sensed to body-sensing and filled
Put.
As the first possible implementation of the third aspect, motion sensing control data packet described in the synchronism output to body
Propagated sensation induction device includes：
The motion sensing control data of M body points position are parsed from the motion sensing control data packet；
The body-sensing sensing device of N types is parsed in the motion sensing control data of i-th of the body points position analytically gone out
Control information；
The body-sensing that the control information parsed is exported to the N types on i-th of body points position respectively senses dress
In putting；
The M, the N and the i are the integer more than or equal to 1, and the i is less than or equal to M.
With reference to the first possible implementation of the third aspect, in second of possible implementation, the synchronization
The motion sensing control data packet to body-sensing sensing device is exported to further include：
If in the motion sensing control data packet, the corresponding control of body-sensing sensing device of the jth type of i-th of body points position
Information processed is preset characters in each data bit, then while the relevant picture frame of the motion sensing control data packet plays,
Stop exporting the control information；
The j is the integer more than or equal to 1, and the j is less than or equal to N.
It is described default in the third possible implementation with reference to second of possible implementation of the third aspect
Character is 0.
Fourth aspect, an embodiment of the present invention provides a kind of real-time output device of multi-medium data, described device includes：
First receiving unit, for during watching video live broadcast, live terminal to be received by net cast platform
The picture frame sent in real time；
Second receiving unit, if being associated with motion sensing control data packet for described image frame, is put down by the net cast
Platform receives the motion sensing control data packet that the live terminal is synchronously sent, and the motion sensing control data packet is by the live end
End instructs according to motion sensing control during net cast and generates；
Output unit, for exporting described image frame to video play device, and motion sensing control data described in synchronism output
Wrap to body-sensing sensing device.
As the first possible implementation of fourth aspect, the output unit includes：
First parsing subelement, for parsing the motion sensing control of M body points position from the motion sensing control data packet
Data；
Second parses subelement, and N species is parsed in the motion sensing control data for i-th of the body points position analytically gone out
The control information of the body-sensing sensing device of type；
First output control subelement, for exporting respectively the control information parsed to i-th of body points position
On N types body-sensing sensing device in；
The M, the N and the i are the integer more than or equal to 1, and the i is less than or equal to M.
With reference to the first possible implementation of fourth aspect, in second of possible implementation, the output
Unit further includes：
Second output control subelement, if in the motion sensing control data packet, the jth species of i-th of body points position
The corresponding control information of body-sensing sensing device of type is preset characters in each data bit, then in the motion sensing control data packet
While relevant picture frame plays, stop exporting the control information；
The j is the integer more than or equal to 1, and the j is less than or equal to N.
It is described default in the third possible implementation with reference to second of possible implementation of fourth aspect
Character is 0.
The embodiment of the present invention can experience the body-sensing to match with net cast content while watching video live broadcast
Simulation, drastically increase online scene can be interactive.
Brief description of the drawings
To describe the technical solutions in the embodiments of the present invention more clearly, below will be to embodiment or description of the prior art
Needed in attached drawing be briefly described, it should be apparent that, drawings in the following description be only the present invention some
Embodiment, for those of ordinary skill in the art, without having to pay creative labor, can also be according to these
Attached drawing obtains other attached drawings.
Fig. 1 is that the real-time sending method of multi-medium data provided in an embodiment of the present invention realizes flow chart；
Fig. 2 is the specific implementation flow chart of the real-time sending method S102 of multi-medium data provided in an embodiment of the present invention；
Fig. 3 is the structure diagram of the real-time dispensing device of multi-medium data provided in an embodiment of the present invention；
Fig. 4 is a kind of schematic diagram of calculate node provided in an embodiment of the present invention；
Fig. 5 is that the real-time output method of multi-medium data provided in an embodiment of the present invention realizes flow chart；
Fig. 6 is the specific implementation flow chart of the real-time output method S503 of multi-medium data provided in an embodiment of the present invention；
Fig. 7 is the structure diagram of the real-time output device of multi-medium data provided in an embodiment of the present invention；
Fig. 8 is the schematic diagram of another calculate node provided in an embodiment of the present invention.
Embodiment
In being described below, in order to illustrate rather than in order to limit, it is proposed that such as tool of particular system structure, technology etc
Body details, to understand thoroughly the embodiment of the present invention.However, it will be clear to one skilled in the art that these are specific
The present invention can also be realized in the other embodiments of details.In other situations, omit to well-known system, device, electricity
Road and the detailed description of method, in case unnecessary details hinders description of the invention.
The real-time Transmission scheme of multi-medium data mentioned by the embodiment of the present invention, can during net cast,
It is inserted into motion sensing control data at any time in live video stream, and motion sensing control data are straight by video with live video stream
Broadcast platform and be transferred to user terminal, so that can to experience body-sensing at the same time during watching video live broadcast anti-for user terminal
Feedback.In embodiments of the present invention, the motion sensing control data be used for body-sensing sensing device is controlled, motion sensing control data with
The form of motion sensing control signal is sent from motion sensing control device, and is sent to the body-sensing sensing dress being electrically connected with motion sensing control device
In putting.In actual product form, the body-sensing sensing device of motion sensing control device and one or more can be with wearable product
It is combined, for example, motion sensing control device and multiple body-sensing sensing devices are attached in the wearable body of parcel user's whole body,
Motion sensing control data are exported by motion sensing control device respectively to multiple body-sensing sensing devices, so that this multiple body-sensing sensing
Device carries out proprioceptive simulation according to the motion sensing control data being respectively received to user.
First, the real-time sending method of multi-medium data provided in an embodiment of the present invention is described in detail, it is realized
Flow is as shown in Figure 1：
In S101, during net cast, instruction monitoring is carried out to live terminal.
Herein, net cast refers mainly to be carried out using internet and stream media technology live, and live terminal can be with
Refer to the acquisition terminal for being used for gathering picture signal and voice signal, can be the audio-video collection video camera of specialty, can also be
Smart mobile phone, tablet etc. possess the intelligent terminal of camera function.Net cast process, generally refers to audio and video herein
The image and audio digital signal that acquisition terminal collects generate live file by streaming coding device, and pass through net cast
Platform is transmitted to user terminal.
In embodiments of the present invention, when live terminal is in net cast process, instruction monitoring is carried out to live terminal,
To detect whether user has assigned motion sensing control instruction by live terminal.
In S102, when listen to the live terminal generate motion sensing control instruction when, referred to according to the motion sensing control
Order generation motion sensing control data packet, and the picture frame for marking the live terminal currently to collect.
During net cast, live terminal provides the operation for assigning motion sensing control instruction on its display interface
Menu, user can assign motion sensing control by the actions menu at any time and instruct, and body-sensing control is carried in motion sensing control instruction
Data processed, for being controlled to body-sensing sensing device.Motion sensing control instruction according to listening to generates motion sensing control data packet,
And one picture frame of mark in the live video stream that live terminal collects, the picture frame are monitored for motion sensing control instruction
When the picture frame that is collected.
Further, as one embodiment of the present of invention, one or more can be carried in each motion sensing control data packet
The motion sensing control data of a body points position, and the motion sensing control data of wherein each body points position are comprising one or more types
Body-sensing sensing device control information.Specifically, as shown in Fig. 2, the process of generation sensing control data packet is as follows：
S201, obtains the motion sensing control data of M body points position, and the motion sensing control data of each body points position include N
The control information of the body-sensing sensing device of type.
The motion sensing control data of the M body points position got are write the motion sensing control data packet by S202.
Wherein, the M and N is the integer more than or equal to 1.
Based on the embodiment shown in Fig. 2, when M and N is equal to 1, each motion sensing control data packet is used in the picture frame
During broadcasting, a kind of body-sensing sensing device on a body points position is controlled to produce proprioceptive simulation；When M, which is more than 1 and N, is equal to 1, often
A motion sensing control data packet is used for when the picture frame plays, and controls the same body-sensing sensing device on multiple body points positions same
When produce proprioceptive simulation；When M and N are all higher than 1, each motion sensing control data packet is used to control a variety of on multiple body points positions
Body-sensing sensing device produces proprioceptive simulation at the same time.
In embodiments of the present invention, for each body points position, according to product demand, can at least integrate as listed in table 1
The body-sensing sensing device for any one or more type lifted：
Table 1
Preferably, in embodiments of the present invention, it is also proposed that a kind of frame format of motion sensing control data packet, the frame format
Form the starting control frame and body-sensing data frame included as frame head.
(1) the control data that some bytes are carried in control frame are originated：
Exemplarily, as shown in table 2, the control data of 4 bytes are included in the starting control frame, are respectively control frame
Byte 1, control frame byte 2, control frame byte 3 and control frame byte 4, each control frame byte can be according to the need of motion sensing control
Corresponding data content is write, for example, can show body points that motion sensing control data packet is covered with control frame byte
Bit quantity, or show whether the motion sensing control data packet needs controlled device to be shielded.
Table 2
(2) the motion sensing control data for the several body-sensing sensor being sequentially written in are carried in body-sensing data frame, at the same time also
Data bit has been reserved, has needed to write the motion sensing control data of other some body-sensing sensors according to exploitation for future：
In body-sensing data frame, the corresponding motion sensing control data of every kind of body-sensing sensor may be used to indicate that the body-sensing senses
The relevant motion sensing control pattern of device, or realized for describing the relevant body-sensing function of the body-sensing sensor.Exemplarily, such as table
Shown in 3, every kind of body-sensing sensor corresponds to the motion sensing control data of 4 bytes, is respectively motion sensing control pattern byte 1, body-sensing work(
Can byte 1, body-sensing function byte 2 and body-sensing function byte 3.
Table 3
Table 4 shows the frame structure example of a complete motion sensing control data packet, as can be seen from Table 4, the body-sensing
The frame head for controlling data packet is the starting control frame of 4 bytes, and 13 kinds of body-sensing sensing devices have been sequentially written in body-sensing data frame
Motion sensing control data, while further include 7 kinds of functions and reserve, the motion sensing control data of 7 kinds of body-sensing sensing devices can be write again, and
The motion sensing control data of every kind of body-sensing sensing device are 4 bytes.
Table 4
Exemplarily, table 5 shows that 13 kinds of body-sensings that table 1 refers to sense preposition corresponding motion sensing control data frame structure：
Table 5
In the communication process of body-sensing sensing data, motion sensing control data are generated according to frame format described above
Bag, can be on the one hand unified data format by the motion sensing control Data Integration of a variety of different types of body-sensing sensing devices,
Propagated sensation induction device fragmentation caused by from different suppliers is effectively eliminated, on the other hand, by a variety of differences
The motion sensing control data of the body-sensing sensing device of type write in a data packet and are transmitted, and can also drop to a certain extent
The appearance of packet loss phenomenon in low data transmission procedure, improves the reliability of data communication.
In addition, on the basis of motion sensing control data packet frame form described above, an implementation as the present invention
Example, during motion sensing control data packet is generated：
If the control information of the body-sensing sensing device of the jth type of i-th of body points position is not got, in the body-sensing
The corresponding each data bit of the control information writes preset characters in control data packet；
The i and j is the integer more than or equal to 1, and the i is less than or equal to the M, the j be less than or
Equal to the N.
That is, a character is preset, if the proprioceptive simulation of a certain type is not matched to some body points position, in motion sensing control
In data packet, each data bit occupied by the body-sensing sensing device of the corresponding the type in body points position is write this and is preset
Character.Need to carry out force feedback for example, the motion sensing control instruction that user assigns only indicates, without carrying out other body-sensing moulds
Intend, then, will be all in addition to the data bit occupied by device for force feedback on all body points positions in motion sensing control data packet
Data bit writes the preset characters.
The frame writing scheme of above motion sensing control data packet can ensure the data length one of all motion sensing control data packets
Cause, so, for each motion sensing control data packet of generation, data length all same, contributes in data check process
In rapidly find packet drop.
Alternatively, the preset characters are 0.
In S103, the motion sensing control data packet is associated with described image frame.
Specifically the motion sensing control data packet generated in S102 is associated with the picture frame marked in S102, should
Incidence relation once it is determined that can then exist always, even after net cast process terminates, by user terminal from video
When whole live process is played back on live platform, which exists all the time, with ensure proprioceptive simulation during live and
All the time identical live video stream progress is all corresponded in replayed section.
In S104, described image frame is sent to the user terminal in real time by net cast platform, at the same will with it is described
The associated motion sensing control data packet of picture frame is synchronously sent to the user terminal by the net cast platform, so that
For the user terminal while described image frame to video play device is exported, motion sensing control data packet described in synchronism output is extremely
Body-sensing sensing device.
In embodiments of the present invention, it is real-time Transmission to regarding for net cast stream and motion sensing control data packet
Frequently live platform (server), then be broadcast to by net cast platform the user terminal of all watching video live broadcasts, therefore,
, also will be with the associated motion sensing control of the picture frame while picture frame is sent to the user terminal in real time by net cast platform
Data packet gives the user terminal of all watching video live broadcasts, so, user by same net cast platform synchronized broadcast
Terminal can synchronously carry out proprioceptive simulation in watching video live broadcast while user terminal plays net cast stream,
Truly to restore the motion sensing control of live terminal user intention.
Corresponding to the real-time sending method of the multi-medium data described in foregoing embodiments, Fig. 3 shows the embodiment of the present invention
The structure diagram of the real-time dispensing device of the multi-medium data of offer.For convenience of description, illustrate only related to the present embodiment
Part.
With reference to Fig. 3, which includes：
Monitoring unit 31, during net cast, instruction monitoring is carried out to live terminal；
Generation unit 32, when listen to the live terminal generate motion sensing control instruction when, according to the motion sensing control
Instruction generation motion sensing control data packet, and the picture frame for marking the live terminal currently to collect；
Associative cell 33, the motion sensing control data packet is associated with described image frame；
Transmitting element 34, described image frame is sent to the user terminal in real time by net cast platform, while will be with institute
The associated motion sensing control data packet of picture frame is stated synchronously to send to the user terminal by the net cast platform, with
Make the user terminal while described image frame to video play device is exported, motion sensing control data packet described in synchronism output
To body-sensing sensing device.
Alternatively, the generation unit 31 includes：
Subelement is obtained, obtains the motion sensing control data of M body points position, the motion sensing control data of each body points position are equal
The control information of body-sensing sensing device comprising N types；
The motion sensing control data of the M body points position got are write the body-sensing by the first write-in subelement
Control data packet；
The M and N is the integer more than or equal to 1.
Alternatively, the generation unit 31 further includes；
Second write-in subelement, if not getting the control of the body-sensing sensing device of the jth type of i-th of body points position
Information, the corresponding each data bit of the control information writes preset characters in the motion sensing control data packet；
The i and j is the integer more than or equal to 1, and the i is less than or equal to the M, the j be less than or
Equal to the N.
Alternatively, the preset characters are 0.
Corresponding to the real-time sending method of the multi-medium data described in foregoing embodiments, Fig. 4 shows the embodiment of the present invention
A kind of schematic diagram of the calculate node 400 provided.For convenience of description, it illustrate only part related to the present embodiment.
Wherein, calculate node 400 is probably the host server comprising computing capability, or personal computer PC, or
Person is portable portable computer or terminal etc., and the specific embodiment of the invention is not done the specific implementation of calculate node
Limit.Calculate node 400 includes：
Processor (processor) 410, communication interface (Communications Interface) 420, memory
(memory) 430, bus 440.
Processor 410, communication interface 420, memory 430 complete mutual communication by bus 440.
Communication interface 420, for communicating with network element, such as Virtual Machine Manager center, shared storage etc..
Processor 410, for executive program.
Specifically, program can include program code, and said program code includes computer-managed instruction.
Processor 410 is probably a central processing unit (Central Processing Unit, CPU), or specific
Integrated circuit (Application Specific Integrated Circuit, ASIC), or be arranged to implement this hair
One or more integrated circuits of bright embodiment.
Memory 430, for storing program.Memory 430 may include high-speed RAM memory, it is also possible to further include non-
Volatile memory (non-volatile memory), for example, at least a magnetic disk storage.
Program specifically can be used for the real-time sending method for performing a kind of multi-medium data, the described method includes：
During net cast, instruction monitoring is carried out to live terminal；
When listen to the live terminal generate motion sensing control instruction when, according to the motion sensing control instruct generation body-sensing
Control data packet, and the picture frame for marking the live terminal currently to collect；
The motion sensing control data packet is associated with described image frame；
Described image frame is sent to the user terminal in real time by net cast platform, while will be associated with described image frame
The motion sensing control data packet by the net cast platform synchronously send to the user terminal so that the user is whole
While described image frame to video play device is exported, motion sensing control data packet described in synchronism output to body-sensing, which senses, to be filled at end
Put.
Alternatively, it is described to instruct generation motion sensing control data packet to include according to the motion sensing control：
The motion sensing control data of M body points position are obtained, the motion sensing control data of each body points position include N types
Body-sensing sensing device control information；
The motion sensing control data of the M body points position got are write into the motion sensing control data packet；
The M and N is the integer more than or equal to 1.
Alternatively, it is described to instruct generation motion sensing control data packet to further include according to the motion sensing control；
If the control information of the body-sensing sensing device of the jth type of i-th of body points position is not got, in the body-sensing
The corresponding each data bit of the control information writes preset characters in control data packet；
The i and j is the integer more than or equal to 1, and the i is less than or equal to the M, the j be less than or
Equal to the N.
Alternatively, the preset characters are 0.
The embodiment of the present invention can be generated and transmitted to user terminal while net cast from live terminal in real time
Motion sensing control data, so that user terminal experiences the body-sensing to match with net cast content while watching video live broadcast
Simulation, drastically increase online scene can be interactive.
Next, the real-time output method of multi-medium data provided in an embodiment of the present invention is described in detail, in fact
Existing flow is as shown in Figure 5：
In S501, during watching video live broadcast, live terminal is received by net cast platform and is sent in real time
Picture frame.
In S502, if described image frame is associated with motion sensing control data packet, institute is received by the net cast platform
The motion sensing control data packet that live terminal is synchronously sent is stated, the motion sensing control data packet is by the live terminal in video
Instructed and generated according to motion sensing control in live process.
In S503, output described image frame is to video play device, and motion sensing control data packet described in synchronism output is extremely
Body-sensing sensing device.
In embodiments of the present invention, user terminal is by the way of watching video live broadcast in the prior art, from net cast
Platform receives the picture frame that live terminal is sent in real time, and as described in embodiment above, if live terminal user is sometime
Motion sensing control instruction has been assigned, then will can have been collected according to the motion sensing control data packet of sensing control instruction generation with the moment
Picture frame user terminal is synchronously sent to by net cast platform, then user terminal output the picture frame to video broadcast
While putting device, also synchronism output motion sensing control data packet to body-sensing sensing device.The life of above-mentioned motion sensing control data packet
Elaborated in foregoing embodiments into method, data format and data content, this will not be repeated here.
As one embodiment of the present of invention, when containing the N kinds on M body points position in each motion sensing control data packet
During the control information of the body-sensing sensing device of type, then as shown in fig. 6, motion sensing control data packet described in the synchronism output is to body
Propagated sensation induction device includes：
S601, parses the motion sensing control data of M body points position from the motion sensing control data packet.
S602, the body-sensing sensing dress of N types is parsed in the motion sensing control data of i-th of the body points position analytically gone out
The control information put.
S603, the control information parsed is exported to the body-sensing biography of the N types on i-th of body points position respectively
In induction device.
The M, the N and the i are the integer more than or equal to 1, and the i is less than or equal to M.
, can be accurate during motion sensing control device exports motion sensing control data packet based on the corresponding embodiments of Fig. 6
Ground exports motion sensing control data in a specific body-sensing sensing device of a specific body points position, then for whole body M
M*N individuals propagated sensation induction device altogether on a body points position, can realize body at the same time during relevant picture frame plays
Feel the output of simulation, bring comprehensive body-sensing to experience.
In addition, as described above, a character is preset on the basis of above-mentioned motion sensing control data packet frame form, if
The proprioceptive simulation of a certain type is not matched to some body points position, then it is in the motion sensing control data packet, the body points position is right
Each data bit occupied by the body-sensing sensing device of the type answered writes the preset characters, for example, write-in preset characters
0, to ensure that the data length of all motion sensing control data packets is consistent.So under the program, body described in the synchronism output
Sensing control data packet to body-sensing sensing device further includes：
If in the motion sensing control data packet, the corresponding control of body-sensing sensing device of the jth type of i-th of body points position
Information processed is preset characters in each data bit, then while the relevant picture frame of the motion sensing control data packet plays,
Stop exporting the control information；
The j is the integer more than or equal to 1, and the j is less than or equal to N.
That is, if motion sensing control device detects that certain specific class body-sensing sensing device on some specific body points position is corresponding
It is preset characters in control information each data bit occupied in body-sensing data packet, then stops at the moment to the body
Body-sensing sensing device output control information on point position, so, although the relevant motion sensing control data of each picture frame
The data length of bag is identical, but motion sensing control device still can judge according to the data content in each data bit
It is that output control information is also off output control information, so as to neatly realize to more body points positions, multiple body-sensings sensing
The output control of device.
The embodiment of the present invention can be generated and transmitted to user terminal while net cast from live terminal in real time
Motion sensing control data, so, user terminal can be experienced and net cast content phase while watching video live broadcast
Matched proprioceptive simulation, drastically increase online scene can be interactive.
Corresponding to the real-time output method of the multi-medium data described in foregoing embodiments, Fig. 7 shows the embodiment of the present invention
The structure diagram of the real-time output device of the multi-medium data of offer.For convenience of description, illustrate only related to the present embodiment
Part.
With reference to Fig. 7, which includes：
First receiving unit 71, during watching video live broadcast, it is real to receive live terminal by net cast platform
When the picture frame that sends；
Second receiving unit 72, if described image frame is associated with motion sensing control data packet, passes through the net cast platform
The motion sensing control data packet that the live terminal is synchronously sent is received, the motion sensing control data packet is by the live terminal
Instructed and generated according to motion sensing control during net cast；
Output unit 73, output described image frame to video play device, and motion sensing control data packet described in synchronism output
To body-sensing sensing device.
Alternatively, the output unit 73 includes：
First parsing subelement, parses the motion sensing control data of M body points position from the motion sensing control data packet；
Second parses subelement, and N types are parsed in the motion sensing control data of i-th of the body points position analytically gone out
The control information of body-sensing sensing device；
First output control subelement, the control information parsed is exported to the N on i-th of body points position respectively
In the body-sensing sensing device of type；
The M, the N and the i are the integer more than or equal to 1, and the i is less than or equal to M.
Alternatively, the output unit 73 further includes：
Second output control subelement, if in the motion sensing control data packet, the jth type of i-th of body points position
The corresponding control information of body-sensing sensing device is preset characters in each data bit, then related in the motion sensing control data packet
Picture frame play while, stop export the control information；
The j is the integer more than or equal to 1, and the j is less than or equal to N.
Alternatively, the preset characters are 0.
Corresponding to the real-time output method of the multi-medium data described in foregoing embodiments, Fig. 8 shows the embodiment of the present invention
A kind of schematic diagram of the calculate node 800 provided.For convenience of description, it illustrate only part related to the present embodiment.
Wherein, calculate node 800 is probably the host server comprising computing capability, or personal computer PC, or
Person is portable portable computer or terminal etc., and the specific embodiment of the invention is not done the specific implementation of calculate node
Limit.Calculate node 800 includes：
Processor (processor) 810, communication interface (Communications Interface) 820, memory
(memory) 830, bus 840.
Processor 810, communication interface 820, memory 830 complete mutual communication by bus 840.
Communication interface 820, for communicating with network element, such as Virtual Machine Manager center, shared storage etc..
Processor 810, for executive program.
Specifically, program can include program code, and said program code includes computer-managed instruction.
Processor 810 is probably a central processing unit (Central Processing Unit, CPU), or specific
Integrated circuit (Application Specific Integrated Circuit, ASIC), or be arranged to implement this hair
One or more integrated circuits of bright embodiment.
Memory 830, for storing program.Memory 830 may include high-speed RAM memory, it is also possible to further include non-
Volatile memory (non-volatile memory), for example, at least a magnetic disk storage.
Program specifically can be used for the real-time output method for performing a kind of multi-medium data, the described method includes：
During watching video live broadcast, the picture frame that live terminal sends in real time is received by net cast platform；
If described image frame is associated with motion sensing control data packet, the live terminal is received by the net cast platform
The motion sensing control data packet synchronously sent, the motion sensing control data packet by the live terminal net cast process
Middle instructed according to motion sensing control generates；
Described image frame is exported to video play device, and motion sensing control data packet described in synchronism output is sensed to body-sensing and filled
Put.
Alternatively, motion sensing control data packet described in the synchronism output to body-sensing sensing device includes：
The motion sensing control data of M body points position are parsed from the motion sensing control data packet；
The body-sensing sensing device of N types is parsed in the motion sensing control data of i-th of the body points position analytically gone out
Control information；
The body-sensing that the control information parsed is exported to the N types on i-th of body points position respectively senses dress
In putting；
The M, the N and the i are the integer more than or equal to 1, and the i is less than or equal to M.
Alternatively, motion sensing control data packet described in the synchronism output to body-sensing sensing device further includes：
If in the motion sensing control data packet, the corresponding control of body-sensing sensing device of the jth type of i-th of body points position
Information processed is preset characters in each data bit, then while the relevant picture frame of the motion sensing control data packet plays,
Stop exporting the control information；
The j is the integer more than or equal to 1, and the j is less than or equal to N.
Alternatively, the preset characters are 0.
It is apparent to those skilled in the art that for convenience of description and succinctly, only with above-mentioned each work(
Can unit division progress for example, in practical application, can be as needed and by above-mentioned function distribution by different functions
Unit is completed, i.e., the internal structure of device is divided into different functional units or module, with complete it is described above whole or
Person's partial function.Each functional unit in embodiment can be integrated in a processing unit or unit is independent
Be physically present, can also two or more units integrate in a unit, above-mentioned integrated unit can both use hard
The form of part is realized, can also be realized in the form of SFU software functional unit.In addition, the specific name of each functional unit is also
For the ease of mutually distinguishing, the protection domain of the application is not limited to.The specific work process of unit in above device, can
With with reference to the corresponding process in aforementioned means embodiment, details are not described herein.
Those of ordinary skill in the art may realize that each exemplary list described with reference to the embodiments described herein
Member and algorithm steps, can be realized with the combination of electronic hardware or computer software and electronic hardware.These functions are actually
Performed with hardware or software mode, application-specific and design constraint depending on technical solution.Professional technician
Described function can be realized using different device to each specific application, but this realization is it is not considered that exceed
The scope of the present invention.
In embodiment provided by the present invention, it should be understood that disclosed device and device, can pass through others
Mode is realized.For example, device embodiment described above is only schematical, for example, the division of module or unit, only
For a kind of division of logic function, there can be other dividing mode when actually realizing, such as multiple units or component can combine
Or another system is desirably integrated into, or some features can be ignored, or do not perform.Another, shown or discussed phase
Coupling or direct-coupling or communication connection between mutually can be by some interfaces, the INDIRECT COUPLING or communication of device or unit
Connection, can be electrical, machinery or other forms.
The unit illustrated as separating component may or may not be physically separate, be shown as unit
Component may or may not be physical location, you can with positioned at a place, or can also be distributed to multiple networks
On unit.Some or all of unit therein can be selected to realize the purpose of this embodiment scheme according to the actual needs.
In addition, each functional unit in each embodiment of the present invention can be integrated in a processing unit, can also
That unit is individually physically present, can also two or more units integrate in a unit.Above-mentioned integrated list
Member can both be realized in the form of hardware, can also be realized in the form of SFU software functional unit.
If integrated unit is realized in the form of SFU software functional unit and is used as independent production marketing or in use, can
To be stored in a computer read/write memory medium.Based on such understanding, the technical solution essence of the embodiment of the present invention
On all or part of the part that contributes in other words to the prior art or the technical solution can be with the shape of software product
Formula embodies, which is stored in a storage medium, including some instructions are used so that a calculating
Machine equipment (can be personal computer, server, or network equipment etc.) or processor perform each reality of the embodiment of the present invention
Apply all or part of step of a device.And foregoing storage medium includes：USB flash disk, mobile hard disk, read-only storage, deposit at random
Access to memory, magnetic disc or CD etc. are various can be with the medium of store program codes.
The above embodiments are merely illustrative of the technical solutions of the present invention, rather than its limitations；Although with reference to the foregoing embodiments
The present invention is described in detail, it will be understood by those of ordinary skill in the art that：It still can be to foregoing each implementation
Technical solution described in example is modified, or carries out equivalent substitution to which part technical characteristic；And these modification or
Replace, the essence of appropriate technical solution is departed from the spirit and scope of each embodiment technical solution of the embodiment of the present invention.
The foregoing is merely a prefered embodiment of the invention, is not intended to limit the invention, it is all the present invention spirit and
All any modification, equivalent and improvement made within principle etc., should all be included in the protection scope of the present invention.
Claims (16)
- A kind of 1. real-time sending method of multi-medium data, it is characterised in that the described method includes：During net cast, instruction monitoring is carried out to live terminal；When listen to the live terminal generate motion sensing control instruction when, according to the motion sensing control instruct generation motion sensing control Data packet, and the picture frame for marking the live terminal currently to collect；Motion sensing control number is carried in motion sensing control instruction According to for being controlled to body-sensing sensing device；The motion sensing control data packet is associated with described image frame；Described image frame is sent to the user terminal in real time by net cast platform, while will be with the associated institute of described image frame State motion sensing control data packet synchronously to send to the user terminal by the net cast platform, so that the user terminal exists While exporting described image frame to video play device, motion sensing control data packet described in synchronism output to body-sensing sensing device.
- 2. the method as described in claim 1, it is characterised in that described that generation motion sensing control number is instructed according to the motion sensing control Include according to bag：The motion sensing control data of M body points position are obtained, the motion sensing control data of each body points position include the body of N types The control information of propagated sensation induction device；The motion sensing control data of the M body points position got are write into the motion sensing control data packet；The M and N is the integer more than or equal to 1.
- 3. method as claimed in claim 2, it is characterised in that described that generation motion sensing control number is instructed according to the motion sensing control Further included according to bag；If the control information of the body-sensing sensing device of the jth type of i-th of body points position is not got, in the motion sensing control The corresponding each data bit of the control information writes preset characters in data packet；The i and j is the integer more than or equal to 1, and the i is less than or equal to the M, and the j is less than or equal to The N.
- 4. method as claimed in claim 3, it is characterised in that the preset characters are 0.
- A kind of 5. real-time output method of multi-medium data, it is characterised in that the described method includes：During watching video live broadcast, the picture frame that live terminal sends in real time is received by net cast platform；If described image frame is associated with motion sensing control data packet, it is synchronous that the live terminal is received by the net cast platform The motion sensing control data packet sent, the motion sensing control data packet is by live terminal root during net cast Instruct and generate according to motion sensing control；Motion sensing control data are carried in motion sensing control instruction, for being carried out to body-sensing sensing device Control；Described image frame is exported to video play device, and motion sensing control data packet described in synchronism output is to body-sensing sensing device.
- 6. method as claimed in claim 5, it is characterised in that motion sensing control data packet described in the synchronism output to body-sensing passes Induction device includes：The motion sensing control data of M body points position are parsed from the motion sensing control data packet；The control of the body-sensing sensing device of N types is parsed in the motion sensing control data of i-th of the body points position analytically gone out Information；The control information parsed is exported into the body-sensing sensing device of the N types on i-th of body points position respectively；The M, the N and the i are the integer more than or equal to 1, and the i is less than or equal to M.
- 7. method as claimed in claim 6, it is characterised in that motion sensing control data packet described in the synchronism output to body-sensing passes Induction device further includes：If in the motion sensing control data packet, the corresponding control letter of body-sensing sensing device of the jth type of i-th of body points position Breath is preset characters in each data bit, then while the relevant picture frame of the motion sensing control data packet plays, stops Export the control information；The j is the integer more than or equal to 1, and the j is less than or equal to N.
- 8. the method for claim 7, it is characterised in that the preset characters are 0.
- 9. the real-time dispensing device of a kind of multi-medium data, it is characterised in that described device includes：Monitoring unit, for during net cast, instruction monitoring to be carried out to live terminal；Generation unit, for when listen to the live terminal generate motion sensing control instruction when, referred to according to the motion sensing control Order generation motion sensing control data packet, and the picture frame for marking the live terminal currently to collect；Taken in motion sensing control instruction With motion sensing control data, for being controlled to body-sensing sensing device；Associative cell, for the motion sensing control data packet to be associated with described image frame；Transmitting element, for described image frame to be sent to the user terminal in real time by net cast platform, while will with it is described The associated motion sensing control data packet of picture frame is synchronously sent to the user terminal by the net cast platform, so that For the user terminal while described image frame to video play device is exported, motion sensing control data packet described in synchronism output is extremely Body-sensing sensing device.
- 10. device as claimed in claim 9, it is characterised in that the generation unit includes：Subelement is obtained, for obtaining the motion sensing control data of M body points position, the motion sensing control data of each body points position are equal The control information of body-sensing sensing device comprising N types；First write-in subelement, for the motion sensing control data of the M body points position got to be write the body-sensing Control data packet；The M and N is the integer more than or equal to 1.
- 11. device as claimed in claim 10, it is characterised in that the generation unit further includes；Second writes subelement, if the control of the body-sensing sensing device of the jth type for not getting i-th of body points position Information, the corresponding each data bit of the control information writes preset characters in the motion sensing control data packet；The i and j is the integer more than or equal to 1, and the i is less than or equal to the M, and the j is less than or equal to The N.
- 12. device as claimed in claim 11, it is characterised in that the preset characters are 0.
- 13. the real-time output device of a kind of multi-medium data, it is characterised in that described device includes：First receiving unit, for during watching video live broadcast, it is real-time to receive live terminal by net cast platform The picture frame of transmission；Second receiving unit, if being associated with motion sensing control data packet for described image frame, is connect by the net cast platform The motion sensing control data packet that the live terminal is synchronously sent is received, the motion sensing control data packet is existed by the live terminal Instructed and generated according to motion sensing control during net cast；Motion sensing control data are carried in motion sensing control instruction, are used for Body-sensing sensing device is controlled；Output unit, for exporting described image frame to video play device, and motion sensing control data packet described in synchronism output is extremely Body-sensing sensing device.
- 14. device as claimed in claim 13, it is characterised in that the output unit includes：First parsing subelement, for parsing the motion sensing control data of M body points position from the motion sensing control data packet；Second parses subelement, and N types are parsed in the motion sensing control data for i-th of the body points position analytically gone out The control information of body-sensing sensing device；First output control subelement, for exporting respectively the control information parsed to the N on i-th of body points position In the body-sensing sensing device of type；The M, the N and the i are the integer more than or equal to 1, and the i is less than or equal to M.
- 15. device as claimed in claim 14, it is characterised in that the output unit further includes：Second output control subelement, if in the motion sensing control data packet, the jth type of i-th of body points position The corresponding control information of body-sensing sensing device is preset characters in each data bit, then related in the motion sensing control data packet Picture frame play while, stop export the control information；The j is the integer more than or equal to 1, and the j is less than or equal to N.
- 16. device as claimed in claim 15, it is characterised in that the preset characters are 0.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611058946.9A CN106534142B (en) | 2016-11-22 | 2016-11-22 | The live transmission method and device of multi-medium data |
| PCT/CN2017/086495 WO2018095003A1 (en) | 2016-11-22 | 2017-05-29 | Method and apparatus for real-time transmission of multimedia data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611058946.9A CN106534142B (en) | 2016-11-22 | 2016-11-22 | The live transmission method and device of multi-medium data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106534142A CN106534142A (en) | 2017-03-22 |
| CN106534142B true CN106534142B (en) | 2018-04-20 |
Family
ID=58357196
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611058946.9A Active CN106534142B (en) | 2016-11-22 | 2016-11-22 | The live transmission method and device of multi-medium data |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN106534142B (en) |
| WO (1) | WO2018095003A1 (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106534142B (en) * | 2016-11-22 | 2018-04-20 | 包磊 | The live transmission method and device of multi-medium data |
| CN107491172B (en) * | 2017-08-16 | 2020-10-09 | 歌尔科技有限公司 | Somatosensory data acquisition method and device and electronic equipment |
| CN109644294B (en) * | 2017-12-29 | 2020-11-27 | 腾讯科技（深圳）有限公司 | Live broadcast sharing method, related equipment and system |
| CN109840016A (en) * | 2018-12-14 | 2019-06-04 | 维沃移动通信有限公司 | A kind of virtual interacting method and terminal device |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101828382A (en) * | 2007-10-16 | 2010-09-08 | 伊梅森公司 | Haptic effect data in the media transport stream synchronously |
| CN102598085A (en) * | 2009-08-14 | 2012-07-18 | K12股份有限公司 | Systems and methods for producing, delivering and managing educational material |
| CN103970266A (en) * | 2013-01-24 | 2014-08-06 | 英默森公司 | Haptic sensation recording and playback |
| CN104049749A (en) * | 2013-03-15 | 2014-09-17 | 英默森公司 | Method and apparatus to generate haptic feedback from video content analysis |
| CN104932678A (en) * | 2014-03-21 | 2015-09-23 | 意美森公司 | Systems and methods for converting sensory data to haptic effects |
Family Cites Families (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| ATE457112T1 (en) * | 2005-12-23 | 2010-02-15 | Ambx Uk Ltd | SCRIPT SYNCHRONIZATION VIA FINGERPRINTS DESIGNED FROM A CONTENT STREAM |
| KR101131856B1 (en) * | 2006-11-03 | 2012-03-30 | 엘지전자 주식회사 | Apparatus For Transmitting Broadcast Signal And Method Of Transmitting And Receiving Broadcast Signal Using Same |
| KR101492635B1 (en) * | 2007-10-16 | 2015-02-17 | 한국전자통신연구원 | Sensory Effect Media Generating and Consuming Method and Apparatus thereof |
| JP2012524452A (en) * | 2009-04-15 | 2012-10-11 | エレクトロニクス アンド テレコミュニケーションズ リサーチ インスチチュート | Method and apparatus for providing metadata for sensory effect, computer-readable recording medium on which metadata for sensory effect is recorded, sensory reproduction method and apparatus |
| US9386063B2 (en) * | 2011-09-19 | 2016-07-05 | Comcast Cable Communications, Llc | Content storage and identification |
| US20150054727A1 (en) * | 2013-08-23 | 2015-02-26 | Immersion Corporation | Haptically enabled viewing of sporting events |
| CN104780403A (en) * | 2014-03-28 | 2015-07-15 | 方小祥 | System and method for providing somatosensory video based on script |
| US9667907B2 (en) * | 2014-09-13 | 2017-05-30 | Vicente Diaz | System and method for haptic integration and generation in multimedia devices |
| US9812165B2 (en) * | 2014-12-19 | 2017-11-07 | Immersion Corporation | Systems and methods for recording haptic data for use with multi-media data |
| US20160366450A1 (en) * | 2015-06-12 | 2016-12-15 | Immersion Corporation | Broadcast haptics architectures |
| CN106534142B (en) * | 2016-11-22 | 2018-04-20 | 包磊 | The live transmission method and device of multi-medium data |
-
2016
- 2016-11-22 CN CN201611058946.9A patent/CN106534142B/en active Active
-
2017
- 2017-05-29 WO PCT/CN2017/086495 patent/WO2018095003A1/en active Application Filing
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101828382A (en) * | 2007-10-16 | 2010-09-08 | 伊梅森公司 | Haptic effect data in the media transport stream synchronously |
| CN102598085A (en) * | 2009-08-14 | 2012-07-18 | K12股份有限公司 | Systems and methods for producing, delivering and managing educational material |
| CN103970266A (en) * | 2013-01-24 | 2014-08-06 | 英默森公司 | Haptic sensation recording and playback |
| CN104049749A (en) * | 2013-03-15 | 2014-09-17 | 英默森公司 | Method and apparatus to generate haptic feedback from video content analysis |
| CN104932678A (en) * | 2014-03-21 | 2015-09-23 | 意美森公司 | Systems and methods for converting sensory data to haptic effects |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106534142A (en) | 2017-03-22 |
| WO2018095003A1 (en) | 2018-05-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106534142B (en) | The live transmission method and device of multi-medium data | |
| CN101174332B (en) | Method, device and system for interactively combining real-time scene in real world with virtual reality scene | |
| CN104796448B (en) | The data processing method and device of network system | |
| CN107835971A (en) | Based on user's tactual space（HapSpace）The method and apparatus that touch feedback and interaction are provided | |
| CN107092353B (en) | Hand touch parameter acquisition and simulation restoration system and method | |
| CN109981711A (en) | Document dynamic play method, apparatus, system and computer readable storage medium | |
| CN108632676A (en) | Display methods, device, storage medium and the electronic device of image | |
| CN107422862A (en) | A kind of method that virtual image interacts in virtual reality scenario | |
| CN108134964A (en) | Net cast stage property stacking method, computer equipment and storage medium | |
| CN107728905A (en) | A kind of barrage display methods, device and storage medium | |
| CN106527730B (en) | Generation, output control method and the device of motion sensing control data | |
| US9378574B2 (en) | Apparatus and method for producing makeup avatar | |
| CN110221759A (en) | A kind of element dragging method, device, storage medium and interactive intelligent tablet computer | |
| CN107261490A (en) | Realize intelligent toy interactive method, client and intelligent toy | |
| CN107948702A (en) | Synchronous method, device, terminal and the storage medium of Application Status | |
| CN106980382A (en) | A kind of method, mobile terminal and the VR equipment of the control of VR device plays | |
| JP2023540785A (en) | Tactile scene representation format | |
| CN114510150A (en) | Experience system of virtual digital world | |
| CN106454365B (en) | Coding, coding/decoding method and the coding of multi-medium data, decoding apparatus | |
| CN107888663A (en) | A kind of method of distribution of document, equipment and computer-readable medium | |
| CN106547357B (en) | The communication processing method and device of body-sensing sensing data | |
| CN207752477U (en) | A kind of Landscape design system based on VR technologies | |
| CN108733216A (en) | A kind of enterprise rollout transaction platform based on three-dimensional exclusive business card | |
| CN108737487A (en) | Method of data synchronization and device, storage medium and electronic device | |
| CN109039851A (en) | Interaction data processing method, device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |