WO2018095001A1

WO2018095001A1 - Method and device for communication processing of sensed somatic sensation data

Info

Publication number: WO2018095001A1
Application number: PCT/CN2017/086493
Authority: WO
Inventors: 包磊
Original assignee: 包磊
Priority date: 2016-11-22
Filing date: 2017-05-29
Publication date: 2018-05-31
Also published as: CN106547357A; CN106547357B

Abstract

The invention is applicable to the field of multimedia technology, and provides a method and device for communication processing of somatic sensation control data. The method comprises: for each image frame of a video file, generating a somatic sensation control data packet related to the image frame; associating all the generated somatic sensation control data packets with the video file; and by means of the association, synchronously outputting the somatic sensation control data packets to a somatic sensation sensing device whilst playing back the video file. The present invention generates somatic sensation control data packets on the basis of each image frame of the video file, and associates the generated somatic sensation control data packets with the video file, thereby being able to synchronously output the somatic sensation control data packets related to the image frame whilst playing back the video file in practical applications, simply and efficiently implementing the somatic sensation simulation matching the virtual environment created by the video content, saving a large amount of time used by post technical processing.

Description

Communication processing method and device for somatosensory sensing data

Technical field

The invention belongs to the field of multimedia technology, and in particular relates to a communication processing method and device for somatosensory sensing data.

Background technique

The prior art can simulate other body sensations that people have, including simulating the perceptions of all human beings such as touch, force, temperature, humidity, wind, and smell. The realization principle of the somatosensory sensing is to send a somatosensory control signal to the somatosensory sensing device attached to the human body surface to control the somatosensory sensing device to perform various somatosensory simulations on the human body. For example, when shaking hands with a virtual character in a VR game, the force feedback control signal is used to control the magnitude and duration of the force generated by the force sensor located at the hand, thereby bringing a force feedback experience that matches the handshake behavior in the game. However, during the development process, the technician found that the prior art has at least the following technical defects:

Existing video content providers and various types of somatosensory device suppliers provide services independently, and the format of video files and somatosensory sensing control signals are also different, so to generate a virtual environment that matches the video content. The somatosensory experience requires a large amount of post-technical processing for the combination or simultaneous output of the two, which consumes a lot of time and cost.

Summary of the invention

In view of this, the embodiments of the present invention provide a communication processing method and apparatus for somatosensory sensing data, so as to solve the somatosensory experience in the prior art that matches the virtual environment created by the video content, a combination of the two is needed. Or synchronous output for a large number of post-technical processing, which will cost a lot of time and cost.

In a first aspect, an embodiment of the present invention provides a communication processing method for a somatosensory control data, where the method includes:

For each image frame of the video file, a somatosensory control data packet associated with the image frame is separately generated;

Associating all of the generated somatosensory control data packets with the video file;

Through the association, the somatosensory control data packet is synchronously outputted to the somatosensory sensing device when the video file is played.

As a first possible implementation manner of the first aspect, the generating a somatosensory control data packet related to the image frame separately includes:

Obtaining somatosensory control data of M body points associated with the image frame, and the somatosensory control data of each body point includes control information of N types of somatosensory sensing devices;

And acquiring the acquired somatosensory control data of the M body points into the somatosensory control data packet related to the image frame;

The M and the N are each an integer greater than or equal to 1.

In conjunction with the first possible implementation of the first aspect, in a second possible implementation, the generating the somatosensory control data packet associated with the image frame separately includes:

If the control information of the jth type of the somatosensory sensing device of the i-th body point associated with the image frame is not obtained, a preset character is written in each data bit corresponding to the control information;

The i and the j are integers greater than or equal to 1, and the i is less than or equal to the M, and the j is less than or equal to the N.

In conjunction with the second possible implementation of the first aspect, in a third possible implementation, the preset character is 0.

As a fourth possible implementation manner of the first aspect, the associating all the generated somatosensory control data packets with the video file includes:

Adding a frame structure to the image frame;

A somatosensory sensor packet associated with the image frame is written in an increased frame structure.

As a fifth possible implementation of the first aspect, all of the somatosensory controls to be generated The association of the data packet with the video file includes:

And all the generated somatosensory control data packets are sequentially arranged according to a playing sequence of the related image frames to generate a somatosensory control file;

An association is established for the video file and the somatosensory control file.

In conjunction with the fifth possible implementation of the first aspect, as a sixth possible implementation, the associating the video file with the sensible control file includes:

The same naming operation is performed on the video file and the somatosensory control file.

In a second aspect, an embodiment of the present invention provides a communication processing apparatus for somatosensory control data, where the apparatus includes:

a generating unit, configured to generate a somatosensory control data packet associated with the image frame for each image frame of the video file;

An association unit, configured to associate all generated somatosensory control data packets with the video file;

And an output unit, configured to synchronously output the somatosensory control data packet to the somatosensory sensing device when playing the video file by using the association.

As a first possible implementation manner of the second aspect, the generating unit includes:

Obtaining a subunit for acquiring somatosensory control data of M body points associated with the image frame, and the somatosensory control data of each body point includes control information of N types of somatosensory sensing devices;

a first writing subunit, configured to write the acquired somatosensory control data of the M body points to the somatosensory control data packet related to the image frame;

The M and the N are each an integer greater than or equal to 1.

With reference to the first possible implementation of the second aspect, in a second possible implementation, the generating unit further includes:

a second writing subunit, configured to: if the control information of the jth type of the somatosensory sensing device of the i-th body point associated with the image frame is not obtained, each data bit corresponding to the control information Write preset characters;

In conjunction with the second possible implementation of the second aspect, in a third possible implementation, the preset character is 0.

As a fourth possible implementation manner of the second aspect, the associating unit includes:

Adding a subunit for adding a frame structure to the image frame;

And a third write subunit for writing a somatosensory sensor data packet associated with the image frame in the added frame structure.

As a fifth possible implementation manner of the second aspect, the associating unit includes:

Generating a subunit, configured to sequentially arrange all the generated somatosensory control data packets according to a play order of the related image frames to generate a somatosensory control file;

An association subunit is configured to establish an association between the video file and the somatosensory control file.

With reference to the fifth possible implementation of the second aspect, in a sixth possible implementation, the associated sub-unit is specifically configured to:

The embodiment of the present invention generates a somatosensory control data packet based on each image frame of the video file, and associates the generated somatosensory control data packet with the video file, so that in an actual application, the image frame can be synchronously outputted when the video file is played. The somatosensory control data packet realizes the somatosensory simulation matching the virtual environment created by the video content simply and efficiently, saving the time cost caused by a large number of post-technical processing.

DRAWINGS

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the embodiments or the description of the prior art will be briefly described below. It is obvious that the drawings in the following description are only the present invention. For some embodiments, other drawings may be obtained from those of ordinary skill in the art in light of the inventive workability.

1 is a flowchart of an implementation of a communication processing method for somatosensory control data according to an embodiment of the present invention;

2 is a specific implementation flowchart of a communication processing method S101 of the somatosensory control data according to an embodiment of the present invention;

3 is a specific implementation flowchart of a communication processing method S102 for receiving somatosensory control data according to an embodiment of the present invention;

4 is a flowchart of a specific implementation of a communication processing method S101 for receiving somatosensory control data according to another embodiment of the present invention;

FIG. 5 is a structural block diagram of a communication processing apparatus for somatosensory control data according to an embodiment of the present invention; FIG.

FIG. 6 is a schematic diagram of a computing node according to an embodiment of the present invention.

detailed description

In the following description, for purposes of illustration and description However, it will be apparent to those skilled in the art that the present invention may be practiced in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the invention.

In the embodiment of the present invention, the somatosensory control data is used to control the somatosensory sensing device, and the somatosensory control data is sent from the somatosensory control device in the form of a somatosensory control signal and transmitted to the somatosensory sensor electrically connected to the somatosensory control device. In the device. In an actual product form, the somatosensory control device and one or more somatosensory sensing devices can be combined with the wearable product, for example, attaching the somatosensory control device and the plurality of somatosensory sensing devices to the wearable body that wraps the entire body of the user. The somatosensory control device outputs the somatosensory control data to the plurality of somatosensory sensing devices, so that the plurality of somatosensory sensing devices perform a somatosensory simulation on the user based on the received somatosensory control data.

FIG. 1 is a flowchart showing an implementation process of a communication processing method for a somatosensory control data according to an embodiment of the present invention, which is described in detail as follows:

In S101, for each image frame of the video file, a body associated with the image frame is separately generated Sense control packet.

The video file refers to a multimedia file that includes audio and video information. Further, the video file may further include a VR video file, which is characterized by being capable of presenting a 3D and full-view visual effect. For the video file formats in the prior art, such as AVI, WMV, MPEG, MOV, etc., when the video information in the video file is acquired, the original content is captured and acquired one frame at a time. Therefore, playing a video file is actually displaying the captured image frame frame by frame, and combining the synchronized play of the audio to achieve an audiovisual effect. Therefore, based on the data characteristics that the video file is substantially composed of image frames, in the embodiment of the present invention, each image frame in the video file is added as a data packet to generate a somatosensory control associated with the image frame. data pack.

Further, as an embodiment of the present invention, the somatosensory control data packet associated with an image frame may carry the somatosensory control data of one or more body points, and the somatosensory control data of each body point includes one Control information for one or more types of somatosensory sensing devices. Specifically, as shown in FIG. 2, the process of generating an image frame related somatosensory control data packet is as follows:

S201: Acquire somatosensory control data of M body points related to the image frame, and the somatosensory control data of each body point includes control information of N types of somatosensory sensing devices.

S202. Write the acquired somatosensory control data of the M body points into the somatosensory control data packet related to the image frame.

Wherein M and the N are integers greater than or equal to 1.

Based on the embodiment shown in FIG. 2, when both M and N are equal to 1, the somatosensory control data packet associated with the image frame is used to control the generation of a somatosensory sensing device on a body point during playback of the image frame. Somatosensory simulation; when M is greater than 1 and N is equal to 1, the somatosensory control data packet associated with each image frame is used to control the same somatosensory sensing device on multiple body points while the image frame is being played. Simulation; when both M and N are greater than 1, the somatosensory control data packet associated with each image frame is used to control a plurality of somatosensory sensing devices on a plurality of body points to simultaneously generate a somatosensory simulation during playback of the image frame.

In the embodiment of the present invention, for each body point, according to product requirements, at least Any one or more types of somatosensory sensing devices listed in Table 1:

Table 1

序号Serial number	体感传感装置Somatosensory sensor	序号Serial number	体感传感装置Somatosensory sensor
11	按摩/振动Massage/vibration	88	风wind
22	触觉Tactile	99	湿润Wet
33	刺痛Stinging	1010	外设灯Peripheral light
44	温暖/热Warm/hot	1111	外设风湿Peripheral rheumatism
55	冷cold	1212	外设风冷Peripheral air cooling
66	气味odor	1313	机械外骨骼Mechanical exoskeleton
77	收缩shrink

Preferably, in the embodiment of the present invention, a frame format of a somatosensory control data packet is also proposed, and the frame format comprises a start control frame as a frame header and a somatosensory data frame.

(1) The control data carrying several bytes in the initial control frame:

Exemplarily, as shown in Table 2, the start control frame includes 4 bytes of control data, which are control frame byte 1, control frame byte 2, control frame byte 3, and control frame byte. 4. Each control frame byte can be written into the corresponding data content according to the needs of the somatosensory control. For example, the control frame byte can be used to indicate the number of body points covered by the somatosensory control data packet, or to indicate the somatosensory control data packet. Whether it needs to be shielded by the somatosensory control device.

Table 2

(2) The somatosensory data frame carries the somatosensory control data of several kinds of somatosensory sensors sequentially written, and also reserves the data bits for writing the somatosensory control data of several other somatosensory sensors according to development needs in the future:

In the somatosensory data frame, the somatosensory control data corresponding to each of the somatosensory sensors can be used to indicate the body The sensor-related somatosensory control mode is used to describe the somatosensory function associated with the somatosensory sensor. Except as shown in Table 3, each of the somatosensory sensors corresponds to 4 bytes of somatosensory control data, which are respectively a somatosensory control mode byte 1, a somatosensory function byte 1, a somatosensory function byte 2, and a somatosensory function byte 3 .

table 3

Table 4 shows an example of the frame structure of a complete somatosensory control data packet. As can be seen from Table 4, the frame header of the somatosensory control data packet is a 4-byte initial control frame, and the somatosensory data frame is sequentially written. The somatosensory control data of 13 kinds of somatosensory sensing devices, including 7 kinds of function reservations, can further write the somatosensory control data of the seven kinds of somatosensory sensing devices, and the somatosensory control data of each somatosensory sensing device is 4 byte.

Table 4

Illustratively, Table 5 shows the somatosensory control data frame structures corresponding to the 13 somatosensory sensing preambles mentioned in Table 1:

table 5

In the communication processing process of the somatosensory sensing data, if the somatosensory control data packet is generated by using the frame format described above, on the one hand, the somatosensory control data of a plurality of different types of somatosensory sensing devices can be integrated into a unified data format. Effectively eliminating the fragmentation phenomenon of the sensing device due to different suppliers; on the other hand, writing the somatosensory control data of a plurality of different types of somatosensory sensing devices into one data packet for transmission, To some extent, the occurrence of packet loss during data transmission is reduced, and the reliability of data communication is improved.

Further, on the basis of the above-described somatosensory control packet frame format, as an embodiment of the present invention, in the process of generating a somatosensory control packet:

That is, a character is preset. For any image frame, if a certain body image is not matched to a certain body shape simulation while the image frame is being played, in the image frame related somatosensory control data packet. Each data bit occupied by the body sensing device corresponding to the body point is written to the preset character. For example, in a virtual indoor environment, no wind sense simulation is required, and each data bit occupied by the wind sensing device on all body points is written in the image frame related somatosensory control data packet. The preset character.

The frame writing scheme of the above-mentioned somatosensory control data packet can ensure that the data lengths of all the somatosensory control data packets are consistent, so that the data lengths of the related somatosensory control data packets are the same for all the image frames in the video file, which is beneficial to the whole The amount of data of the somatosensory control file associated with the video file is estimated and can help to quickly detect the packet loss during the data verification process.

Optionally, the preset character is 0.

In S102, all the generated somatosensory control data packets are associated with the video file.

In the above S101, for each image frame of the video file, a somatosensory control data packet associated with the image frame is generated, and all the somatosensory control data packets to be generated are generated after all the somatosensory control data packets are generated. Associate with the video file.

The following describes how the somatosensory control packet is associated with the video file:

As an embodiment of the present invention, as shown in FIG. 3, the associating all the generated somatosensory control data packets with the video file includes:

S301. Add a frame structure to the image frame.

S302. Write a somatosensory sensor data packet related to the image frame in an increased frame structure.

For a video file, although it is equivalent to a frame set of image frames, for a sequence of captured image frames, it needs to be video-encoded by a certain video encoding format to generate a video encoded file, and finally encapsulated into Video files of different formats, in the process of video encoding, each image frame is not seamlessly connected, but some reference frames or synchronization frames are generated according to requirements of different video encoding formats, and therefore, in the present invention In an embodiment, on one hand, the frame structure may be added to the image frame itself, or, on the other hand, the frame structure may be added to the reference frame, the synchronization frame or other types of frames associated with the image frame, and the frame is added. The somatosensory sensing data packet associated with the image frame is written in the structure, thereby realizing the association between the somatosensory control data packet and the video file by modifying the video encoding mode.

As another embodiment of the present invention, as shown in FIG. 4, the associating all the generated somatosensory control data packets with the video file includes:

S401: Arranging all the generated somatosensory control data packets in order according to the playing order of the related image frames to generate a somatosensory control file.

S402. Establish an association between the video file and the somatosensory control file.

As described above, the process of playing a video file is actually a process of sequentially displaying the captured image frames on a frame-by-frame basis. Therefore, in the embodiment of the present invention, all generated according to the playback order of the image frames in the video file. The somatosensory control data packets are sequentially arranged to generate a somatosensory control file, and an association between the video file and the somatosensory control file is established.

Based on the embodiment corresponding to FIG. 4, the establishing the association between the video file and the sensible control file may be implemented by:

That is, the video file and the somatosensory control file of the same file name are regarded as files having an associated relationship, and when the somatosensory control is performed, the somatosensory control file associated with the video file can be recognized and read by the file name.

In S103, the somatosensory control data packet is synchronously outputted to the somatosensory sensing device when the video file is played by the association.

After the association between all the somatosensory control data packets and the video file is completed, the somatosensory control device of the somatosensory product can simultaneously output the somatosensory control data packet related to the image frame to the somatosensory sensing device while the video file is being played. To achieve synchronization between the somatosensory simulation and the virtual environment.

Corresponding to the communication processing method of the somatosensory control data described in the above embodiments, FIG. 5 is a block diagram showing the structure of the communication processing apparatus of the somatosensory control data provided by the embodiment of the present invention. For the convenience of explanation, only the parts related to the present embodiment are shown.

Referring to Figure 5, the apparatus includes:

a generating unit 51, for each image frame of the video file, respectively generating a somatosensory control data packet related to the image frame;

The associating unit 52 associates all the generated somatosensory control data packets with the video file;

The output unit 53 synchronously outputs the somatosensory control data packet to the somatosensory sensing device when the video file is played by the association.

Optionally, the generating unit 51 includes:

Obtaining a subunit, acquiring somatosensory control data of M body points associated with the image frame, and the somatosensory control data of each body point includes control information of N types of somatosensory sensing devices;

a first writing subunit, wherein the acquired somatosensory control data of the M body points are written into the somatosensory control data packet related to the image frame;

The M and the N are each an integer greater than or equal to 1.

Optionally, the generating unit 51 further includes:

a second writing subunit, if the control information of the jth type of the somatosensory sensing device of the i-th body point associated with the image frame is not obtained, each data bit corresponding to the control information is written Preset character;

Optionally, the preset character is 0.

Optionally, the associating unit 52 includes:

Adding a subunit, adding a frame structure to the image frame;

A third write subunit that writes a somatosensory sensor packet associated with the image frame in an increased frame structure.

Optionally, the associating unit 52 includes:

Generating a subunit, sequentially arranging all the generated somatosensory control data packets in accordance with a play order of the associated image frames to generate a somatosensory control file;

Optionally, the associated subunit is specifically configured to:

Corresponding to the communication processing method of the somatosensory control data described in the above embodiments, FIG. 6 is a schematic diagram of a computing node 600 according to an embodiment of the present invention. For the convenience of explanation, only the parts related to the present embodiment are shown.

The computing node 600 may be a host server that includes computing power, or a personal computer PC, or a portable computer or terminal that can be carried. The specific embodiment of the present invention does not limit the specific implementation of the computing node. Compute node 600 includes:

A processor 610, a communications interface 620, a memory 630, and a bus 640.

The processor 610, the communication interface 620, and the memory 630 complete communication with each other via the bus 640.

The communication interface 620 is configured to communicate with a network element, such as a virtual machine management center, shared storage, and the like.

The processor 610 is configured to execute a program.

In particular, the program can include program code, the program code including computer operating instructions.

The processor 610 may be a Central Processing Unit (CPU), or an Application Specific Integrated Circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention.

The memory 630 is configured to store a program. Memory 630 may include high speed RAM memory and may also include non-volatile memory, such as at least one disk memory.

The program may be specifically configured to perform a communication processing method for a somatosensory control data, the method comprising:

Optionally, the generating a somatosensory control data packet related to the image frame separately includes:

The M and the N are each an integer greater than or equal to 1.

Optionally, the preset character is 0.

Optionally, the associating all the generated somatosensory control data packets with the video file includes:

Adding a frame structure to the image frame;

Optionally, establishing association between the video file and the somatosensory control file includes:

It will be clearly understood by those skilled in the art that, for convenience and brevity of description, only the division of each functional unit described above is exemplified. In practical applications, the above function assignment can be completed by different functional units as needed. The internal structure of the device is divided into different functional units or modules to perform all or part of the functions described above. Each functional unit in the embodiment may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit, and the integrated unit may be implemented in the form of hardware. It can also be implemented in the form of a software functional unit. In addition, the specific names of the respective functional units are only for the purpose of facilitating mutual differentiation, and are not intended to limit the scope of protection of the present application. For the specific working process of the unit in the foregoing device, reference may be made to the corresponding process in the foregoing device embodiment, and details are not described herein again.

Those of ordinary skill in the art will appreciate that the elements and algorithm steps of the various examples described in connection with the embodiments disclosed herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the solution. A person skilled in the art can use different means for implementing the described functions for each particular application, but such implementation should not be considered to be beyond the scope of the present invention.

In the embodiments provided by the present invention, it should be understood that the disclosed apparatus and apparatus may be implemented in other manners. For example, the device embodiments described above are merely illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there may be another division manner, for example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored or not executed. Another point, the coupling or direct coupling or mutual coupling or display or discussion The communication connection may be an indirect coupling or communication connection through some interface, device or unit, and may be in electrical, mechanical or other form.

The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

An integrated unit, if implemented in the form of a software functional unit and sold or used as a standalone product, can be stored in a computer readable storage medium. Based on such understanding, the technical solution of the embodiments of the present invention may contribute to the prior art or all or part of the technical solution may be embodied in the form of a software product stored in a storage. The medium includes a number of instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor to perform all or part of the steps of various embodiments of the embodiments of the present invention. The foregoing storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read only memory, a random access memory, a magnetic disk, or an optical disk.

The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to be limiting; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that The technical solutions are described in the following, and the technical features of the embodiments of the present invention are not deviated from the spirit and scope of the technical solutions of the embodiments of the present invention.

The above are only the preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalents, and improvements made within the spirit and scope of the present invention should be included in the scope of the present invention. Inside.

Claims

A communication processing method for somatosensory control data, characterized in that the method comprises:

For each image frame of the video file, a somatosensory control data packet associated with the image frame is separately generated;

Associating all of the generated somatosensory control data packets with the video file;

Through the association, the somatosensory control data packet is synchronously outputted to the somatosensory sensing device when the video file is played.
The method according to claim 1, wherein said generating a somatosensory control data packet associated with said image frame respectively comprises:

Obtaining somatosensory control data of M body points associated with the image frame, and the somatosensory control data of each body point includes control information of N types of somatosensory sensing devices;

And acquiring the acquired somatosensory control data of the M body points into the somatosensory control data packet related to the image frame;

The M and the N are each an integer greater than or equal to 1.
The method according to claim 2, wherein said generating a somatosensory control data packet associated with said image frame separately comprises;

If the control information of the jth type of the somatosensory sensing device of the i-th body point associated with the image frame is not obtained, a preset character is written in each data bit corresponding to the control information;

The i and the j are integers greater than or equal to 1, and the i is less than or equal to the M, and the j is less than or equal to the N.
The method of claim 3 wherein said predetermined character is zero.
The method of claim 1, wherein the associating the generated all of the somatosensory control data packets with the video file comprises:

Adding a frame structure to the image frame;

A somatosensory sensor packet associated with the image frame is written in an increased frame structure.
The method of claim 1 wherein said said body sense control to be generated The association of the data package with the video file includes:

And all the generated somatosensory control data packets are sequentially arranged according to a playing sequence of the related image frames to generate a somatosensory control file;

An association is established for the video file and the somatosensory control file.
The method of claim 6 wherein establishing an association between the video file and the sensible control file comprises:

The same naming operation is performed on the video file and the somatosensory control file.
A communication processing device for somatosensory control data, characterized in that the device comprises:

a generating unit, configured to generate a somatosensory control data packet associated with the image frame for each image frame of the video file;

An association unit, configured to associate all generated somatosensory control data packets with the video file;

And an output unit, configured to synchronously output the somatosensory control data packet to the somatosensory sensing device when playing the video file by using the association.
The apparatus according to claim 8, wherein said generating unit comprises:

Obtaining a subunit for acquiring somatosensory control data of M body points associated with the image frame, and the somatosensory control data of each body point includes control information of N types of somatosensory sensing devices;

a first writing subunit, configured to write the acquired somatosensory control data of the M body points to the somatosensory control data packet related to the image frame;

The M and the N are each an integer greater than or equal to 1.
The apparatus according to claim 9, wherein said generating unit further comprises:

a second writing subunit, configured to: if the control information of the jth type of the somatosensory sensing device of the i-th body point associated with the image frame is not obtained, each data bit corresponding to the control information Write preset characters;

The i and the j are integers greater than or equal to 1, and the i is less than or equal to the M, and the j is less than or equal to the N.
The apparatus of claim 10 wherein said predetermined character is zero.
The apparatus of claim 8 wherein said associating unit comprises:

Adding a subunit for adding a frame structure to the image frame;

And a third write subunit for writing a somatosensory sensor data packet associated with the image frame in the added frame structure.
The apparatus of claim 8 wherein said associating unit comprises:

Generating a subunit, configured to sequentially arrange all the generated somatosensory control data packets according to a play order of the related image frames to generate a somatosensory control file;

An association subunit is configured to establish an association between the video file and the somatosensory control file.
The device according to claim 13, wherein the associated subunit is specifically configured to:

The same naming operation is performed on the video file and the somatosensory control file.