CN112423020B

CN112423020B - Motion capture data distribution and acquisition method and system

Info

Publication number: CN112423020B
Application number: CN202010378795.5A
Authority: CN
Inventors: 原超; 涂中英; 王炜煜
Original assignee: Shanghai Bilibili Technology Co Ltd
Current assignee: Shanghai Bilibili Technology Co Ltd
Priority date: 2020-05-07
Filing date: 2020-05-07
Publication date: 2022-12-27
Anticipated expiration: 2040-05-07
Also published as: CN112423020A

Abstract

The invention discloses a motion capture data distribution method, which comprises the following steps: detecting whether actions of a user are captured; when the motion is captured, generating motion capture data; encapsulating the motion capture data into corresponding data frames; sending the data frame to a live distribution service, so that the live distribution service forwards the data frame to a VR client. The invention can realize large-scale distribution of motion capture data, and makes it possible for users of multiple VR clients in different local area networks to simultaneously perform virtual reality experience.

Description

Motion capture data distribution and acquisition method and system

Technical Field

The invention relates to the technical field of virtual reality, in particular to a method and a system for distributing and acquiring motion capture data.

Background

In an existing Virtual Reality experience, after a motion capture device acquires motion capture data, the motion capture data is sent to a Virtual Reality (VR) client in the same local area network as the motion capture device. When the number of VR clients is large and the distribution region is wide, the motion capture equipment cannot realize large-scale distribution, and the VR clients in different local area networks with the motion capture equipment cannot acquire motion capture data from the motion capture equipment, and more cannot realize virtual reality experience of users in different regions.

Disclosure of Invention

The invention aims to provide a method, a system, a computer device and a readable storage medium for distributing and acquiring motion capture data, which are used for solving the defect that in the prior art, motion capture data cannot be distributed to multiple VR clients in different local area networks in a large scale by a motion capture device, so that users at the VR clients in different local area networks cannot realize virtual reality experience at the same time.

According to an aspect of the present invention, there is provided a motion capture data distribution method, the method comprising the steps of:

detecting whether actions of a user are captured;

when the motion is captured, generating motion capture data;

encapsulating the motion capture data into corresponding data frames;

forwarding the data frame to a VR client via a live distribution service.

Optionally, the motion capture data distribution method is applied to an electronic device, the electronic device includes at least one motion capture data repeater, and the encapsulating the motion capture data into corresponding data frames includes:

broadcasting the motion capture data to a preset local area network port, so that the motion capture data repeater acquires the motion capture data by monitoring the preset local area network port, and packaging the motion capture data into the data frame by the motion capture data repeater.

Optionally, the motion capture data repeater encapsulates the motion capture data into the data frame, including:

and according to a preset data protocol format, setting the message type of the head of a preset data frame as an audio data packet or a video data packet, and inserting the motion capture data into the preset data frame in a message load mode to obtain the data frame.

Optionally, the method further comprises:

based on TCP, a first command data sender cloud command forwarder and a second command data receiver from the cloud command forwarder are used for establishing a handshake connection with the VR client through the cloud command forwarder so as to establish a communication connection with the VR client and determine software version information of the VR client.

According to another aspect of the present invention, there is also provided a motion capture data acquisition method applied in an electronic device applied in a VR client, the method including the steps of:

acquiring a data frame from a live distribution service;

extracting motion capture data from the data frame according to a preset data protocol format;

and sending the motion capture data to a preset port so that the VR client can acquire the motion capture data from the preset port and render the virtual character according to the motion capture data.

Optionally, the manner of sending the motion capture data to the preset port is a broadcast manner or a unicast manner.

In order to achieve the above object, the present invention also provides a motion capture data distribution system including:

the method comprises the steps that communication connection is established between the motion capture equipment and a VR client;

when the motion capturing equipment captures the motion of a user, the motion capturing equipment is used for generating motion capturing data, packaging the motion capturing data to generate a corresponding data frame, and sending the data frame to a live broadcast distribution service;

the live distribution service is used for forwarding the data frame to the VR client;

the VR client is used for acquiring the data frame, extracting the motion capture data from the data frame, and rendering the virtual character according to the motion capture data.

Optionally, the VR client is specifically configured to:

a motion capture data replayer in the VR client acquires the data frame from the live distribution service, extracts motion capture data from the data frame according to a preset data protocol format, and sends the motion capture data to a preset port;

and the VR client monitors the preset port to acquire the motion capture data from the preset port and renders corresponding motions on the virtual character.

Optionally, the motion capture data distribution system further comprises a cloud command forwarder configured to:

receiving first command data from the motion capture device based on TCP and forwarding the first command data to the VR client;

receive second command data from the VR client and forward the second command data to the motion capture device to enable a communicative connection between the motion capture device and the VR client.

To achieve the above object, the present invention also provides a motion capture data distribution system, including:

the detection module is used for detecting whether actions of the user are captured or not;

a generation module for generating motion capture data when the motion is captured;

the encapsulating module is used for encapsulating the motion capture data into corresponding data frames;

a first sending module, configured to send the data frame to a live distribution service, so that the live distribution service forwards the data frame to a VR client.

To achieve the above object, the present invention also provides a motion capture data acquisition system, including:

an acquisition module for acquiring data frames from a live distribution service;

the extraction module is used for extracting motion capture data from the data frames according to a preset data protocol format;

and the second sending module is used for sending the motion capture data to a preset port so that the VR client can obtain the motion capture data from the preset port and render the virtual character according to the motion capture data.

In order to achieve the above object, the present invention further provides a computer device, which specifically includes: a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the motion capture data distribution method or the motion capture data acquisition method introduced above when executing the computer program.

In order to achieve the above object, the present invention also provides a computer-readable storage medium on which a computer program is stored, which, when executed by a processor, implements the steps of the motion capture data distribution method or the motion capture data acquisition method introduced above.

According to the method, the system, the computer device and the readable storage medium for distributing and acquiring the motion capture data, motion capture data are generated by the motion capture device according to the acquired motion and are broadcasted to a preset local area network port, the motion capture data are acquired from the preset local area network port by a data repeater and are packaged into corresponding data frames, and then the data frames are sent to a live broadcast distribution service, so that the live broadcast distribution service forwards the data frames to a VR client. According to the invention, the data frame comprising the motion capture data is pushed to the live broadcast distribution service, and the live broadcast distribution service distributes the data frame to each VR client in real time, so that captured motions can be rendered on the VR client when the motion capture equipment and the VR client are in different local area networks, thereby realizing large-scale distribution of the motion capture data, and enabling users of a plurality of VR clients in different local area networks to perform virtual reality experience at the same time.

Drawings

Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to refer to like parts throughout the drawings. In the drawings:

FIG. 1 is a diagram of an alternative application environment for a motion capture data distribution method provided by an embodiment of the present disclosure;

FIG. 2 is an alternative flow chart diagram of a motion capture data distribution method provided by an embodiment of the present disclosure;

FIG. 3 is a schematic diagram illustrating an alternative specific flow chart of a method for distributing motion capture data according to an embodiment of the present disclosure;

fig. 4 is a schematic diagram illustrating an alternative specific flowchart of step S104 in fig. 2;

fig. 5 is a schematic diagram of an alternative specific flowchart of step S300 in fig. 4;

FIG. 6 is a diagram of the RTMP data frame header format;

FIG. 7 is an alternative flow chart diagram of a motion capture data acquisition method provided by an embodiment of the present disclosure;

FIG. 8 is an alternative block diagram of a motion capture data distribution system provided by an embodiment of the present disclosure;

FIG. 9 is a schematic diagram of an alternative program module of the motion capture data distribution system provided by embodiments of the present disclosure;

FIG. 10 is a schematic diagram of an alternative program module of the motion capture data acquisition system provided by the embodiments of the present disclosure;

fig. 11 is a schematic diagram of an alternative hardware architecture of a computer device according to an embodiment of the present disclosure.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. All other embodiments, which can be obtained by a person skilled in the art without making any creative effort based on the embodiments in the present invention, belong to the protection scope of the present invention.

The motion capture data distribution method according to the present invention will be described below with reference to the drawings.

FIG. 1 is a diagram of an alternative application environment for the motion capture data distribution method of the present invention. Fig. 1 includes a motion capture device and software (hereinafter, referred to as a motion capture device), a VR client device and software (hereinafter, referred to as a VR client), a universal live broadcast distribution service, and a cloud command repeater, where the motion capture device includes a motion capture data repeater (abbreviated as a repeater in fig. 1), and the VR client includes a motion capture data repeater (abbreviated as a repeater in fig. 1). The motion capture device is used for capturing the motion of the user and forwarding the generated motion data to the VR client through a universal live distribution service. The VR client and the motion capture device may be in the same local area network, or may be in different local area networks. The cloud command forwarder establishes a communication connection between the motion capture device and the VR client by forwarding command data of the motion capture device and the VR client to each other before the motion capture device forwards the motion data to a general live distribution service.

It should be noted that, the general live distribution service refers to a live distribution scheme, and when the push end pushes a data stream to the live distribution service, the live distribution service distributes the data stream to each connected client in real time.

Fig. 2 is a schematic flow chart of an alternative method for distributing motion capture data according to the present invention. It is to be understood that the flowchart in the embodiment of the present method is not intended to limit the order of executing steps, and is applied to an electronic device, and the following description is made by taking the motion capture device in fig. 1 as an executing subject. The motion capture device may include a mobile terminal such as a mobile phone, a tablet computer, a notebook computer, a palm top computer, a Personal Digital Assistant (PDA), a Portable Media Player (PMP), a navigation device, a wearable device, a smart band, a pedometer, and the like, and a fixed terminal such as a Digital TV, a desktop computer, and the like.

As shown in fig. 2, the method specifically includes the following steps:

step S100: detecting whether the motion of the user is captured.

Specifically, after the motion capture device establishes a communication connection with the VR client, the motion capture device detects whether the motion of the user is captured at any moment. The device for capturing the motion of the user may be a camera, an infrared ray, a motion detector, etc., and is not limited herein.

In an exemplary embodiment, as shown in fig. 3, the method further includes step S200:

based on a Transmission Control Protocol (TCP), a first command data sender is connected with a cloud command repeater and receives second command data from the cloud command repeater, so that handshake connection with the VR client is established through the cloud command repeater, communication connection with the VR client is established, and software version information of the VR client is determined.

Specifically, with reference to fig. 1, first, the motion capture device forwards the first command data to the VR client by forwarding the first command data to a cloud command forwarder based on TCP. Then, after receiving the first command data, the VR client confirms the first command data and forwards second command data to the cloud command forwarder, and the cloud command forwarder forwards the second command data to the motion capture device. And finally, the motion capture device receives the second command data and forwards third command data to the VR client through the cloud command forwarder, so that handshake connection between the motion capture device and the VR client is realized, namely communication connection between the motion capture device and the VR client is realized.

In practical applications, the cloud command forwarder may determine whether it is needed based on the provider of the motion capture data. If a provider provides a complete set of lan solutions, but when the motion capture data is transmitted, in order to prevent the own technology from being stolen by others due to commercial confidentiality, the implementation needs to be performed through a handshake connection, that is, the cloud command repeater in this case is equivalent to a process of performing a handshake. If a provider needs a VR client to send information such as heartbeat to the motion capture device, in the unicast mode, in order to reduce the data transmission load of the motion capture device, the VR client needs to be implemented by the cloud command repeater. However, for some providers, the transmission of motion capture data can be implemented without handshaking, that is, a cloud command repeater is not required.

It should be noted that, in the process of handshaking connection between the motion capture device and the VR client, the software version information of the motion capture device and the VR client is determined, so as to provide guarantee for normal communication between the motion capture device and the VR client. The cloud command forwarder is implemented as a simple TCP/UDP forwarder. According to the method, the two-way forwarding of the command data is realized through the cloud command forwarder, and the smooth communication between the motion capture equipment and the VR client of each local area network is realized.

Step S102: when the motion is captured, motion capture data is generated.

Step S104: encapsulating the motion capture data into corresponding data frames. The data frame may be a data frame of a Streaming media Protocol (HTTP Live Streaming, HLS), a Real-Time Transport Protocol (RTCP), a Real-Time Streaming Protocol (RTSP), or a Real-Time Messaging Protocol (RTMP) based on HTTP.

Specifically, after generating the motion capture data, the motion capture device further encapsulates the motion capture data into a data frame in a corresponding data format, and then transmits the data frame via TCP.

In an exemplary embodiment, as shown in fig. 4, the step S104 may include a step S300.

Step S300: broadcasting the motion capture data to a preset local area network port, so that the motion capture data repeater acquires the motion capture data by monitoring the preset local area network port, and encapsulating the motion capture data into the data frame by the motion capture data repeater.

Specifically, referring to fig. 1, after the motion capture device establishes a communication connection with the VR client, a motion capture data repeater is started, wherein the motion capture data repeater is applied to the motion capture device. After the motion capture data is generated, the motion capture data is broadcasted to a specific local area network port, the motion capture data repeater monitors the local area network port all the time, and after the motion capture data is monitored, the motion capture data is encapsulated into the data frame (for example, RTMP data frame). According to the method, the action capturing data are forwarded to the preset local area network port, so that when a user performs virtual reality experience, the action capturing equipment is not limited by a single local area network any more, and can be in communication connection with VR client sides of different local area networks.

In an exemplary embodiment, as shown in fig. 5, the step S300 may include a step S400.

Step S400: according to a preset data protocol format, setting a Message Type (Message Type) of a preset data frame header as an Audio Packet (Audio Packet) or a Video Packet (Video Packet), and inserting the motion capture data into the preset data frame in a Message load (Message Payload) form to obtain the data frame.

Specifically, taking the encapsulation of the RTMP data frame as an example, according to the RTMP data protocol format, an RTMP data frame is composed of a data frame Header (Message Header) and a Message Payload (Message Payload), wherein the data frame Header includes: the Timestamp (Timestamp), the Length of the Message Payload (Payload Length), the Type of Message (Message Type), and the Stream ID (Stream ID), and the header format of the data frame can be referred to fig. 6. When motion capture data is encapsulated into an RTMP data frame, if the Message Type is set as an audio data packet, setting the Message Type =0x08; if the Message Type is set to the video packet, the Message Type =0x09 is set. Other fields of the data frame header are configured according to the transmission timing. The motion capture data is then placed into the message payload field, thereby completing encapsulation of the RTMP data frame. The method facilitates the transmission of the motion capture data by encapsulating the motion capture data into data frames.

Step S106: sending the data frame to a live distribution service, so that the live distribution service forwards the data frame to a VR client.

Specifically, please continue to refer to fig. 1, taking a data frame as an example of an RTMP data frame, when the motion capture data repeater encapsulates the motion capture data, the obtained RTMP data frame after encapsulation is sent to a live broadcast distribution service. And after receiving the RTMP data frame, the live broadcast distribution service forwards the RTMP data frame to a VR client.

In practical applications, such as VR concerts, VR clients of users distributed in various regions of various countries can obtain the RTMP data frames from a live distribution service.

According to the embodiment of the invention, the data frame obtained after encapsulation is sent to the live broadcast distribution service, and the live broadcast distribution service further distributes the data frame to each VR client, so that large-scale distribution of motion capture data is realized, and communication with VR clients in each local area network can be realized, so that users of multiple VR clients in different local area networks can simultaneously perform virtual reality experience.

FIG. 7 is a flow chart illustrating an alternative method of motion capture data acquisition. It is to be understood that the flowchart in the embodiment of the method is not intended to limit the order of executing steps, and is applied to an electronic device, and the following description is made by taking the motion capture data player in the VR client in fig. 1 as an example. The VR client may include a mobile terminal such as a mobile phone, a tablet computer, a notebook computer, a palm top computer, a Personal Digital Assistant (PDA), a Portable Media Player (PMP), a navigation device, a wearable device, a smart band, a pedometer, and a fixed terminal such as a Digital TV, a desktop computer, and the like. As shown in fig. 7, the method specifically includes the following steps:

step S500: data frames are obtained from a live distribution service.

Specifically, referring to fig. 1, after a VR client establishes a communication connection with a motion capture device, the VR client starts the motion capture data player, and then the motion capture data player is in communication connection with a live distribution service. And after the motion capture data player establishes communication connection with the live broadcast distribution service, the motion capture data player acquires the data frame obtained after encapsulation from the live broadcast distribution service in real time. The data frame may be a data frame of HLS, RTCP, RTSP, or RTMP, and the embodiment of the present invention is described only by taking an RTMP data frame as an example, that is, the motion capture data replayer acquires the RTMP data frame from the live broadcast distribution service in real time.

In an exemplary embodiment, the step of establishing a communication connection between the VR client and a motion capture device specifically includes: based on TCP, a third command data sender cloud command forwarder and a fourth command data receiver are used for receiving the fourth command data from the cloud command forwarder, so that a handshake connection with the motion capture device is established through the cloud command forwarder, a communication connection with the motion capture device is established, and software version information of the motion capture device is determined.

Step S502: and extracting motion capture data from the data frames according to a preset data protocol format.

Specifically, taking the data frame as an RTMP data frame as an example, according to the format of the RTMP data protocol, an RTMP data frame is composed of a data frame Header (Message Header) and a Message Payload (Message Payload), where the data frame Header includes: a Timestamp (Timestamp), a Length of a Message Payload (Payload Length), a Type of Message (Message Type), and a Stream ID (Stream ID), and an RTMP data frame header format can be referred to fig. 6. The message payload portion is the motion capture data. Motion capture data may be extracted or parsed from the RTMP data frames according to the RTMP data protocol format.

Step S504: and sending the motion capture data to a preset port so that the VR client acquires the motion capture data from the preset port and renders a virtual character according to the motion capture data.

Specifically, after motion capture data is extracted from the RTMP data frame, the motion capture data is sent to a preset port, and then the VR client acquires the motion capture data from the preset port and renders a virtual character according to the motion capture data to control the motion or position of the virtual character. For example: and rendering the corresponding action to the virtual character.

In practical applications, for example: in the VR concert, users may be distributed in various regions of various countries, and the users (i.e., VR clients) distributed in various regions of various countries can perform VR experience by connecting live broadcast distribution service, so that the users at multiple VR clients of various local area networks can perform virtual reality experience at the same time, thereby greatly enhancing the participation of the users.

It should be noted that the manner of sending the motion capture data to the preset port may be a broadcast manner or a unicast manner. The embodiment of the invention can break through the limitation that the motion capture equipment can be in the same local area network with the VR client in the virtual reality experience, and the VR client can also receive the motion capture data captured by the motion capture equipment when the motion capture equipment and the VR client are in different local area networks, so that users in different local area networks can simultaneously perform the virtual reality experience.

FIG. 8 is a schematic diagram of an alternative module of the motion capture data distribution system. The motion capture data distribution system includes: motion capture device, VR customer end, live distribution service and high in the clouds order repeater, wherein:

the motion capture device establishes a communication connection with the VR client.

Specifically, the motion capture device establishes a communication connection with the VR client based on TCP.

In an exemplary embodiment, the method for establishing a communication connection between the motion capture device and the VR client may specifically include: based on TCP, a cloud command forwarder forwards command data of the motion capture device and the VR client in a two-way manner, establishes a handshake connection between the motion capture device and the VR client, establishes a communication connection between the motion capture device and the VR client, and determines software version information of the motion capture device and the VR client.

Specifically, with reference to fig. 1, first, the motion capture device forwards the first command data to the VR client by forwarding the first command data to a cloud command forwarder based on TCP. Then, after receiving the first command data, the VR client confirms the first command data and forwards second command data to the cloud command forwarder, and the command forwarder forwards the second command data to the motion capture device. And finally, the motion capture device receives the second command data and forwards third command data to the VR client through the cloud command forwarder, so that handshake connection between the motion capture device and the VR client is realized, namely communication connection between the motion capture device and the VR client is realized.

It should be noted that, during the handshake connection between the motion capture device and the VR client, the software version information of the motion capture device and the VR client is determined, so as to provide a guarantee for the normal communication between the motion capture device and the VR client. The cloud command forwarder is implemented as a simple TCP/UDP forwarder. According to the method, the cloud command forwarder is used for forwarding the command data in two directions, so that smooth communication between the motion capture equipment and VR clients of all local area networks is realized.

When the motion capture device captures the motion of a user, the motion capture device generates motion capture data, packages the motion capture data to generate a corresponding data frame, and sends the data frame to a live broadcast distribution service. The data frame may be a data frame of HLS, RTCP, RTSP, or RTMP, and the embodiment of the present invention is described by taking an RTMP data frame as an example.

Specifically, after the motion capture device generates motion capture data, the motion capture device further needs to encapsulate the motion capture data into a data frame in a corresponding data format, and then transmit the data frame to a live distribution service through TCP.

In an exemplary embodiment, the motion capture device is configured to broadcast the motion capture data to a predetermined local area network port, wherein the motion capture device comprises a motion capture data repeater. And then the motion capture data repeater monitors the preset local area network port, acquires the motion capture data after monitoring the motion capture data, and encapsulates the motion capture data into the data frame.

Specifically, referring to fig. 1, after the motion capture device establishes a communication connection with the VR client, a motion capture data repeater is started, wherein the motion capture data repeater is applied to the motion capture device. And after the motion capture data is generated, broadcasting the motion capture data to a specific local area network port, monitoring the local area network port by the motion capture data repeater at all times, and after the motion capture data is monitored, encapsulating the motion capture data into the RTMP data frame. According to the method, the action capturing data are forwarded to the preset local area network port, so that when a user performs virtual reality experience, the action capturing equipment is not limited by a single local area network any more, and can be in communication connection with VR client sides of different local area networks.

In an exemplary embodiment, the motion capture data repeater is configured to set a message type of a header of a preset data frame to be an audio data packet or a video data packet according to a preset data protocol format, and insert the motion capture data into the preset data frame in a form of a message payload to obtain the data frame.

Specifically, taking the RTMP data frame as an example, according to the RTMP data protocol format, an RTMP data frame is composed of a data frame Header (Message Header) and a Message Payload (Message Payload), wherein the data frame Header includes: the Timestamp (Timestamp), the Length of the Message Payload (Payload Length), the Type of Message (Message Type), and the Stream ID (Stream ID), and the header format of the data frame can be referred to fig. 6. When motion capture data is encapsulated into an RTMP data frame, if the Message Type is set as an audio data packet, setting the Message Type =0x08; if the Message Type is set to the video packet, the Message Type =0x09 is set. Other fields of the data frame header are configured according to the transmission timing. The motion capture data is then placed into the message payload field, thereby completing the encapsulation of the RTMP data frame. The method facilitates the transmission of motion capture data by encapsulating the motion capture data into data frames.

The live distribution service is to forward the data frames to the VR client.

Specifically, with continued reference to fig. 1, after a VR client establishes a communication link with a motion capture device, the VR client starts the motion capture data replayer, which is then communicatively coupled with a live distribution service. Then, a motion capture data replayer in the VR client obtains the data frame from the live distribution service, extracts motion capture data from the data frame according to a preset data protocol format, and sends the motion capture data to a preset port. For example: and when the motion capture data player acquires the RTMP data frame from the live broadcast distribution service, extracting motion capture data from the RTMP data frame according to an RTMP data protocol format, and sending the motion capture data to a preset port. And finally, the VR client acquires the motion capture data from the preset port by monitoring the preset port, and renders corresponding motion on the virtual character to control the motion or position of the virtual character and the like. For example: and rendering the corresponding action to the virtual character.

The motion capture data distribution system further comprises a cloud command forwarder for:

second command data is received from the VR client and forwarded to the motion capture device to enable a communicative connection between the motion capture device and the VR client.

Wherein the first command data and the second command data include software version information of the motion capture device and the VR client, respectively.

It should be noted that the motion capture data player may send the motion capture data to a predetermined port in a broadcast or unicast manner.

According to the embodiment of the invention, the packaged data frames are sent to the live broadcast distribution service, and the live broadcast distribution service further distributes the data frames to each VR client, so that large-scale distribution of motion capture data is realized, and communication with VR clients in each local area network is realized, so that users of multiple VR clients in different local area networks can simultaneously perform virtual reality experience.

Based on the motion capture data distribution method provided in the above embodiment, a motion capture data distribution system is provided in the present embodiment. In particular, fig. 9 shows an alternative block diagram of the motion capture data distribution system, which is partitioned into one or more program modules, stored in a storage medium, and executed by one or more processors to implement the present invention. The program module referred to in the present invention is a series of computer program instruction segments capable of performing a specific function, and is more suitable for describing the execution process of the motion capture data distribution system in the storage medium than the program itself, and the following description will specifically describe the function of each program module of the present embodiment.

As shown in fig. 9, the motion capture data distribution system specifically includes the following components:

the detecting module 100 is used for detecting whether the motion of the user is captured.

Specifically, after the motion capture device establishes a communication connection with the VR client, the detection module 100 detects whether the motion of the user is captured at any time. The device for capturing the motion of the user may be a camera, an infrared ray, a motion detector, etc., and is not limited herein.

In an exemplary embodiment, the motion capture data distribution system further includes a connection module configured to send and receive first command data to and from the cloud command forwarder a cloud command forwarder based on TCP, to establish a handshake connection with the VR client through the cloud command forwarder, to establish a communication connection with the VR client, and to determine software version information with the VR client.

Specifically, with reference to fig. 1, first, the connection module forwards the first command data to the VR client by forwarding the first command data to a cloud command forwarder based on TCP. Then, after receiving the first command data, the VR client confirms the first command data and forwards second command data to the cloud command forwarder, and the cloud command forwarder forwards the second command data to the motion capture device. Finally, the connection module receives the second command data and forwards a third command data to the VR client through the cloud command forwarder, so that handshake connection between the motion capture device and the VR client is realized, namely communication connection between the motion capture device and the VR client is realized.

In practical applications, the cloud command forwarder may determine whether it is needed based on the provider of the motion capture data. If a provider provides a complete set of lan solutions, but when the motion capture data is transmitted, in order to prevent the own technology from being stolen by others due to commercial confidentiality, the implementation needs to be performed through a handshake connection, that is, the cloud command repeater in this case is equivalent to a process of performing a handshake. If a provider needs a VR client to send information such as heartbeat to the motion capture device, in the unicast mode, the VR client needs to be implemented by the cloud command repeater in order to reduce the data transmission load of the motion capture device. However, for some providers, the transmission of motion capture data can be implemented without handshaking, that is, a cloud command repeater is not required.

It should be noted that, in the process of handshaking connection between the capture device and the VR client, the software version information of the motion capture device and the VR client is determined, so as to provide guarantee for normal communication between the motion capture device and the VR client. The cloud command forwarder is implemented as a simple TCP/UDP forwarder. According to the method, the cloud command forwarder is used for forwarding the command data in two directions, so that smooth communication between the motion capture equipment and VR clients of all local area networks is realized.

A generating module 101, configured to generate motion capture data when the motion is captured.

An encapsulating module 102, configured to encapsulate the motion capture data into corresponding data frames. The data frame may be a data frame of HLS, RTCP, RTSP, or RTMP, and the embodiment of the present invention is described by taking an RTMP data frame as an example.

Specifically, after generating the motion capture data, the encapsulation module 102 further encapsulates the motion capture data into a data frame in a corresponding data format, and then transmits the data frame through TCP.

In an exemplary embodiment, the encapsulation module 102 is further configured to:

broadcasting the motion capture data to a preset local area network port, so that the motion capture data repeater acquires the motion capture data by monitoring the preset local area network port, and packaging the motion capture data into the RTMP data frame.

Specifically, referring to fig. 1, after the motion capture device establishes a communication connection with the VR client, a motion capture data repeater is started, wherein the motion capture data repeater is applied to the motion capture device. After the motion capture data is generated, the motion capture data is broadcasted to a specific local area network port, the motion capture data repeater monitors the local area network port at all times, and after the motion capture data is monitored, the encapsulation module 102 encapsulates the motion capture data into the RTMP data frame. According to the method, the action capturing data are forwarded to the preset local area network port, so that when a user performs virtual reality experience, the action capturing equipment is not limited by a single local area network any more, and can be in communication connection with VR client sides of different local area networks.

In an exemplary embodiment, the encapsulation module 102 may also be used.

According to a preset data protocol format, setting a Message Type (Message Type) of a preset data frame header as an Audio Packet (Audio Packet) or a Video Packet (Video Packet), and inserting the motion capture data into the preset data frame in a Message load (Message Payload) form to obtain the data frame.

A first sending module 103, configured to send the data frame to a live distribution service, so that the live distribution service forwards the data frame to a VR client.

Specifically, please continue to refer to fig. 1, taking a data frame as an RTMP data frame as an example, after the motion capture data repeater encapsulates the motion capture data, the first sending module 103 sends the encapsulated RTMP data frame to a live broadcast distribution service. And after receiving the RTMP data frame, the live broadcast distribution service forwards the RTMP data frame to a VR client.

In practical applications, such as VR concerts, VR clients of users distributed in various regions of various countries can obtain the RTMP data frames from live distribution services.

According to the embodiment of the invention, the data frame obtained after encapsulation is sent to the live broadcast distribution service, and the live broadcast distribution service further distributes the data frame to each VR client, so that large-scale distribution of motion capture data is realized, and communication with VR clients in each local area network is realized, so that users of multiple VR clients in different local area networks can simultaneously perform virtual reality experience.

Based on the motion capture data acquisition method provided in the above embodiment, a motion capture data acquisition system is provided in the present embodiment. In particular, FIG. 10 illustrates an alternative schematic diagram of program modules for the motion capture data acquisition system, which is partitioned into one or more program modules that are stored in a storage medium and executed by one or more processors to implement the present invention. The program modules referred to in the present invention are a series of computer program instruction segments capable of performing specific functions, and are more suitable for describing the execution process of the motion capture data acquisition system in the storage medium than the program itself, and the following description will specifically describe the functions of each program module in this embodiment.

As shown in fig. 10, the motion capture data acquisition system specifically includes the following components:

an obtaining module 200, configured to obtain an RTMP data frame from a live distribution service.

Specifically, referring to fig. 1, after a VR client establishes a communication connection with a motion capture device, the VR client starts the motion capture data replayer, and then the motion capture data replayer is in communication connection with a live distribution service. After the motion capture data replayer establishes a communication connection with the live distribution service, the obtaining module 200 obtains a data frame obtained after encapsulation in real time from the live distribution service. The data frame may be a data frame of HLS, RTCP, RTSP, or RTMP, and the embodiment of the present invention is only described by taking an RTMP data frame as an example, that is, the motion capture data replayer acquires an RTMP data frame from the live broadcast distribution service in real time.

In an exemplary embodiment, the step of establishing a communication connection between the VR client and a motion capture device specifically includes: based on TCP, third command data is sent to a cloud command repeater and fourth command data is received from the cloud command repeater, so that handshake connection with the motion capture device is established through the cloud command repeater, communication connection with the motion capture device is established, and software version information of the motion capture device is determined.

An extracting module 201, configured to extract motion capture data from the data frame according to a preset data protocol format.

Specifically, taking the data frame as an RTMP data frame as an example, according to the format of the RTMP data protocol, an RTMP data frame is composed of a data frame Header (Message Header) and a Message Payload (Message Payload), where the data frame Header includes: timestamp (Timestamp), length of Message Payload (Payload Length), type of Message (Message Type), and Stream ID (Stream ID), RTMP data frame header format can be seen in fig. 6. The message payload portion is the motion capture data. The extraction module 201 may extract or parse motion capture data from the RTMP data frames according to the RTMP data protocol format.

A second sending module 202, configured to send the motion capture data to a preset port, so that the VR client obtains the motion capture data from the preset port, and renders a virtual character according to the motion capture data.

Specifically, after the motion capture data is extracted from the RTMP data frame, the second sending module 202 sends the motion capture data to a preset port, and then the VR client obtains the motion capture data from the preset port and renders the virtual character according to the motion capture data to control the motion or position of the virtual character.

In practical applications, for example: VR concert, the user probably distributes in each region of each country, and the user who distributes in each region of each country (also be VR client) all can be through connecting our server (live distribution service) in order to carry out VR experience, and then makes the user that is in a plurality of VR clients of each LAN can carry out virtual reality experience simultaneously, has greatly strengthened user's sense of participation.

It should be noted that the manner of sending the motion capture data to the preset port may be a broadcast manner or a unicast manner. The embodiment of the invention can break through the limitation that the motion capture equipment and the VR client end need to be in the same local area network in virtual reality experience, and the VR client end can also receive motion capture data captured by the motion capture equipment when the motion capture equipment and the VR client end are in different local area networks, so that users in different local area networks can simultaneously perform virtual reality experience.

The embodiment also provides a computer device, such as a smart phone, a tablet computer, a notebook computer, a desktop computer, a rack server, a blade server, a tower server or a rack server (including an independent server or a server cluster composed of a plurality of servers) capable of executing programs, and the like. As shown in fig. 11, the computer device 30 of the present embodiment includes at least but is not limited to: a memory 301, a processor 302 communicatively coupled to each other via a system bus. It is noted that fig. 11 only shows a computer device 30 with components 301-302, but it is to be understood that not all shown components are required to be implemented, and that more or fewer components may be implemented instead.

In this embodiment, the memory 301 (i.e., the readable storage medium) includes a flash memory, a hard disk, a multimedia card, a card-type memory (e.g., SD or DX memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, and the like. In some embodiments, the storage 301 may be an internal storage unit of the computer device 30, such as a hard disk or a memory of the computer device 30. In other embodiments, the memory 301 may also be an external storage device of the computer device 30, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like provided on the computer device 30. Of course, the memory 301 may also include both internal and external storage units of the computer device 30. In the present embodiment, the memory 301 is generally used for storing an operating system and various types of application software installed on the computer device 30, such as the program codes of the motion capture data distribution system of the above-described embodiment. In addition, the memory 301 may also be used to temporarily store various types of data that have been output or are to be output.

Processor 302 may be a Central Processing Unit (CPU), controller, microcontroller, microprocessor, or other data Processing chip in some embodiments. The processor 302 generally serves to control the overall operation of the computer device 30.

Specifically, in the present embodiment, the processor 302 is configured to execute a program of a motion capture data distribution method stored in the processor 302, and when executed, the program of the motion capture data distribution method implements the following steps:

detecting whether actions of a user are captured;

when the motion is captured, generating motion capture data;

encapsulating the motion capture data into corresponding data frames;

sending the data frame to a live distribution service, so that the live distribution service forwards the data frame to a VR client.

The embodiment also provides a computer device, such as a smart phone, a tablet computer, a notebook computer, a desktop computer, a rack server, a blade server, a tower server or a rack server (including an independent server or a server cluster composed of a plurality of servers) capable of executing programs, and the like. As shown in fig. 11, the computer device 30 of the present embodiment includes at least, but is not limited to: a memory 301, a processor 302, communicatively coupled to each other via a system bus. It is noted that fig. 11 only shows a computer device 30 with components 301-302, but it is to be understood that not all shown components are required to be implemented, and that more or fewer components may be implemented instead.

In this embodiment, the memory 301 (i.e., the readable storage medium) includes a flash memory, a hard disk, a multimedia card, a card-type memory (e.g., SD or DX memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, and the like. In some embodiments, the storage 301 may be an internal storage unit of the computer device 30, such as a hard disk or a memory of the computer device 30. In other embodiments, the memory 301 may also be an external storage device of the computer device 30, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a flash memory Card (FlashCard), or the like provided on the computer device 30. Of course, the memory 301 may also include both internal and external storage devices for the computer device 30. In the present embodiment, the memory 301 is generally used for storing an operating system and various types of application software installed on the computer device 30, such as the program codes of the motion capture data acquisition system of the above-described embodiment. Further, the memory 301 may also be used to temporarily store various types of data that have been output or are to be output.

Processor 302 may be a Central Processing Unit (CPU), controller, microcontroller, microprocessor, or other data Processing chip in some embodiments. The processor 302 generally functions to control the overall operation of the computer device 30.

Specifically, in the present embodiment, the processor 302 is configured to execute a program of a motion capture data acquisition method stored in the processor 302, the computer is applied to a VR client, and the program of the motion capture data acquisition method implements the following steps when executed:

acquiring a data frame from a live distribution service;

For the specific embodiment of the process of the above method steps, reference may be made to the above embodiments, and details are not repeated here.

The present embodiments also provide a computer readable storage medium, such as a flash memory, a hard disk, a multimedia card, a card type memory (e.g., SD or DX memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, a server, an App, etc., having stored thereon a computer program that when executed by a processor implements the method steps of:

detecting whether actions of a user are captured;

when the motion is captured, generating motion capture data;

encapsulating the motion capture data into corresponding data frames;

The present embodiment also provides a computer-readable storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory (e.g., SD or DX memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, a server, an App application mall, etc., having stored thereon a computer program, which is applied in a VR client, the computer program realizing the following method steps when executed by a processor:

acquiring a data frame from a live distribution service;

and sending the motion capture data to a preset port so that the VR client acquires the motion capture data from the preset port and renders a virtual character according to the motion capture data.

For the specific embodiment of the process of the above method steps, reference may be made to the above embodiments, and details of this embodiment are not repeated herein.

In the method, the system, the computer device, and the readable storage medium for distributing motion capture data provided in this embodiment, the motion capture device generates motion capture data according to an acquired motion, and broadcasts the motion capture data to a preset lan port, and the data forwarder acquires the motion capture data from the preset lan port, and encapsulates the motion capture data into corresponding data frames, and then forwards the data frames to a general live broadcast distribution service, so that the live broadcast distribution service forwards the data frames to VR clients in each lan. According to the invention, the data frame comprising the motion capture data is pushed to the live broadcast distribution service, and the live broadcast distribution service distributes the data frame to each VR client in real time, so that captured motions can be rendered on the VR client when the motion capture equipment and the VR client are in different local area networks, thereby realizing large-scale distribution of the motion capture data, and enabling users of a plurality of VR clients in different local area networks to perform virtual reality experience at the same time.

It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrases "comprising a component of' 8230; \8230;" does not exclude the presence of another like element in a process, method, article, or apparatus that comprises the element.

The above-mentioned serial numbers of the embodiments of the present invention are only for description, and do not represent the advantages and disadvantages of the embodiments.

Through the description of the foregoing embodiments, it is clear to those skilled in the art that the method of the foregoing embodiments may be implemented by software plus a necessary general hardware platform, and certainly may also be implemented by hardware, but in many cases, the former is a better implementation.

The above description is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention, and all equivalent structures or equivalent processes performed by the present invention or directly or indirectly applied to other related technical fields are also included in the scope of the present invention.

Claims

1. A method of motion capture data distribution, the method comprising:

detecting whether actions of a user are captured;

when the motion is captured, generating motion capture data;

encapsulating the motion capture data into corresponding data frames;

sending the data frame to a live distribution service so that the live distribution service forwards the data frame to a VR client, wherein the live distribution service refers to a live distribution scheme;

the method further comprises the following steps: establishing a communication connection with the VR client through a cloud command forwarder.

2. The method of claim 1, wherein the motion capture data distribution method is applied in an electronic device, the electronic device including at least one motion capture data repeater, and wherein the encapsulating the motion capture data into corresponding data frames comprises:

3. The method of claim 2, wherein the motion capture data forwarder encapsulates the motion capture data into the data frame, comprising:

and according to a preset data protocol format, setting the message type of a preset data frame head as an audio data packet or a video data packet, and inserting the motion capture data into the preset data frame in a message load mode to obtain the data frame.

4. The method of claim 1, wherein the method further comprises:

based on TCP, sending first command data to a cloud command repeater and receiving second command data from the cloud command repeater so as to establish handshake connection with the VR client through the cloud command repeater, establish communication connection with the VR client and determine software version information of the VR client.

5. A motion capture data acquisition method is applied to an electronic device, wherein the electronic device is applied to a VR client, and the method comprises the following steps:

acquiring a data frame from a live distribution service, wherein the live distribution service refers to a live distribution scheme;

sending the motion capture data to a preset port so that the VR client can acquire the motion capture data from the preset port and render a virtual character according to the motion capture data;

the method further comprises the following steps: and establishing a communication connection with the motion capture device through the cloud command forwarder.

6. The method of claim 5, wherein the manner of sending the motion capture data to the predetermined port is broadcast or unicast.

7. A motion capture data distribution system, the system comprising:

the motion capture equipment establishes communication connection with the VR client through the cloud command forwarder;

when the motion capture equipment captures the motion of a user, the motion capture equipment is used for generating motion capture data, packaging the motion capture data to generate a corresponding data frame, and sending the data frame to live broadcast distribution service, wherein the live broadcast distribution service refers to a live broadcast distribution scheme;

8. The motion capture data distribution system of claim 7, wherein the VR client is specifically configured to:

and the VR client acquires the motion capture data from the preset port by monitoring the preset port and renders corresponding motion on the virtual character.

9. The motion capture data distribution system of claim 7, wherein the cloud command forwarder is to:

10. A motion capture data distribution system, comprising:

the detection module is used for detecting whether actions of a user are captured or not;

a generation module for generating motion capture data when capturing the motion;

a first sending module, configured to send the data frame to a live distribution service, so that the live distribution service forwards the data frame to a VR client, where the live distribution service refers to a live distribution scheme;

the motion capture data distribution system further comprises a connection module for establishing a communication connection with the VR client through a cloud command forwarder.

11. A motion capture data acquisition system, comprising:

the system comprises an acquisition module, a data frame acquisition module and a data frame transmission module, wherein the acquisition module is used for acquiring a data frame from a live broadcast distribution service, and the live broadcast distribution service refers to a live broadcast distribution scheme;

the extraction module is used for extracting motion capture data from the data frame according to a preset data protocol format; the second sending module is used for sending the motion capture data to a preset port so that the VR client can obtain the motion capture data from the preset port and render a virtual character according to the motion capture data;

the motion capture data acquisition system further comprises a communication interface for establishing communication with the motion capture device via the cloud command forwarder.