WO2017076224A1

WO2017076224A1 - User interaction method and system based on virtual reality

Info

Publication number: WO2017076224A1
Application number: PCT/CN2016/103733
Authority: WO
Inventors: 郑少华; 黎剑辉; 张圳; 朱一伟; 罗海彬; 湛浩
Original assignee: 丰唐物联技术（深圳）有限公司
Priority date: 2015-11-05
Filing date: 2016-10-28
Publication date: 2017-05-11
Also published as: CN106681479A

Abstract

A user interaction method and system based on virtual reality: acquiring movement data detected by sensors worn on different positions on the body of a user (S10); on the basis of the positions of the sensors on the body of the user, parsing the acquired movement data to acquire the movement currently executed by the user (S20); feeding back the movement currently executed by the user to a corresponding virtual role in virtual reality, and controlling the corresponding virtual role in virtual reality to execute a corresponding operation (S30); the present invention has the beneficial effect of user limb movements being detected and acquired by means of sensors worn on the body of a user, enabling the user to interact with a virtual role in a virtual scene, thus improving the sense of reality of the virtual reality; furthermore, the user can perceive information about the environment in which the virtual role is located, thus improving the user experience.

Description

Description: Inventive name: User interaction method and system based on virtual reality

[0001] The present invention relates to the field of virtual reality technologies, and in particular, to a user interaction method and system based on virtual reality.

Background technique

[0002] With the rapid development of virtual reality technology, there are more and more products that use virtual reality technology. Virtual reality technology has the following basic characteristics: Virtual reality technology is a computer simulation system that can create and experience a virtual world. It uses a computer to generate a simulation environment. It is an interactive three-dimensional dynamic view of multi-source information fusion. And the system simulation of the entity behavior, so that the user can truly feel and immersed in the environment; therefore, after the user performs activities such as games and activity exercises based on the virtual reality technology, the stereoscopic and realistic feeling of the image displayed on the display screen is required. More intense and more real.

[0003] At present, in the use of virtual reality technology for activities such as games or activity exercises, there is no technical solution for detecting and acquiring the user's physical actions by the sensors worn by the user, thereby interacting with the virtual characters in the virtual scene. It affects the realism of users who use virtual scenes for games or event drills.

technical problem

[0004] In view of the above, it is necessary to provide a virtual reality-based user interaction method and system for detecting and acquiring a user's physical motion through a sensor worn by a user, thereby enabling a user to perform a virtual character in a virtual scene. Interactive, enhance the realism of virtual reality.

Problem solution

Technical solution

[0005] An embodiment of the present invention discloses a virtual reality-based user interaction method, including the following steps: [0006] acquiring action data detected by sensors worn at different positions on a user's body;

[0007] according to the setting position of the sensor on the user's body, parsing the acquired action data, and acquiring an action currently performed by the user; [0008] feedback the currently performed action of the obtained user to the corresponding virtual character in the virtual reality, and control the corresponding virtual character in the virtual reality to perform a corresponding operation.

[0009] Preferably, the virtual reality based user interaction method further includes:

[0010] Obtaining environment information of a scenario in which the virtual character is currently located in the virtual reality, and feeding the environment information to the user, so that the user can perceive the environment information.

[0011] Preferably, the environmental information includes:

[0012] Temperature information, humidity information, air pressure information, wind direction information, and rainfall or snowfall information.

[0013] Preferably, the sensors worn at different positions on the user's body include:

[0014] Sensors respectively disposed on the user's hands, feet, and waist.

[0015] Preferably, the action that is currently performed by the acquired user is fed back to a corresponding virtual character in the virtual reality

And controlling the corresponding virtual character in the virtual reality to perform corresponding operations, including:

[0016] according to the currently performed action of the acquired user, drawing a motion track corresponding to the action currently performed by the user

, obtaining the motion track information after drawing;

[0017] The corresponding virtual character in the virtual reality is controlled to perform a corresponding operation according to the motion trajectory information according to the drawn motion trajectory information.

[0018] Corresponding to a virtual reality-based user interaction method disclosed above, the present invention also discloses a virtual reality-based user interaction system, the system including multiple positions worn on different positions on the user's body. Sensor

[0019] The virtual reality based user interaction system further includes:

[0020] an obtaining module, configured to acquire motion data detected by sensors worn at different positions on the user's body

[0021] The parsing module is configured to parse the acquired action data according to the set position of the sensor on the user's body, and obtain an action currently performed by the user;

[0022] The control module is configured to feed back the currently performed action of the acquired user to a corresponding virtual character in the virtual reality, and control the corresponding virtual character in the virtual reality to perform a corresponding operation.

[0023] Preferably, the virtual reality-based user interaction system further includes:

[0024] The interaction module is configured to obtain environment information of a scenario in which the virtual character is currently located in the virtual reality, and feed the environment information to the user, so that the user can perceive the environment information. [0025] Preferably, the environmental information includes:

[0026] Temperature information, humidity information, air pressure information, wind direction information, and rainfall or snowfall information.

[0027] Preferably, the sensors worn at different positions on the user's body include:

[0028] Sensors respectively disposed on the user's hands, feet, and waist.

[0029] Preferably, the control module comprises:

[0030] a drawing unit, configured to draw a motion trajectory corresponding to the currently performed action of the user according to the obtained action currently performed by the user, and obtain motion trajectory information after the drawing;

[0031] The control unit is configured to control, according to the drawn motion trajectory information, a corresponding virtual character in the virtual reality to perform a corresponding operation according to the motion trajectory information.

Advantageous effects of the invention

Beneficial effect

[0032] A virtual reality-based user interaction method and system of the present invention can achieve the following beneficial effects: [0033] action data detected by acquiring sensors worn at different positions on the user's body; according to the sensor in the user's body Setting the location, parsing the obtained action data, and acquiring the action currently performed by the user; feeding back the currently performed action of the obtained user to the corresponding virtual character in the virtual reality, and controlling the corresponding virtual character execution in the virtual reality Corresponding operation; having the sensor worn by the user to detect and acquire the user's limb movement, thereby enabling the user to interact with the virtual character in the virtual scene, thereby improving the realism of the virtual reality.

Brief description of the drawing

DRAWINGS

1 is a schematic flowchart of an embodiment of a virtual reality-based user interaction method according to the present invention; [0035] FIG. 2 is a virtual reality-based user interaction method according to the present invention, and step S30 in the embodiment of FIG. Schematic diagram of an embodiment of an embodiment;

3 is a schematic flowchart diagram of another embodiment of a virtual reality-based user interaction method according to the present invention; [0037] FIG. 4 is a block diagram of an embodiment of a virtual reality-based user interaction system according to the present invention;

[0038] FIG. 5 is a block diagram of an embodiment of the control module 80 in the embodiment of FIG. 4 in the virtual reality-based user interaction system of the present invention;

6 is a block diagram of another embodiment of a virtual reality based user interaction system of the present invention. [0040] The implementation, functional features and advantages of the embodiments of the present invention will be further described with reference to the accompanying drawings. Embodiments of the invention

[0041] The technical solutions of the present invention are further described below in conjunction with the drawings and specific embodiments. It is understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0042] The present invention provides a virtual reality-based user interaction method and system for detecting and acquiring a user's physical motion through a sensor worn by a user, thereby enabling a user to interact with a virtual character in a virtual scene. Improve the realism of virtual reality.

1 is a schematic flowchart of an implementation manner of a virtual reality-based user interaction method according to the present invention. As shown in FIG. 1, the virtual reality-based user interaction method of the present invention may be implemented as the following steps. S10-S30:

[0044] Step S10: Acquire motion data detected by sensors worn at different positions on the user's body;

[0045] In the embodiment of the present invention, the user wears a corresponding sensor in different positions on the body in advance. When the user moves, the sensor located in different parts of the user can know the user's made by detecting the corresponding data. The corresponding action.

[0046] In a preferred embodiment, the sensors worn at different locations on the user's body include: sensors respectively disposed on the user's hands, feet, and waist. For example, respective sensors are provided on the user's hands, feet, and waist, respectively, so that motion data of the user's hands, feet, and body torso portions can be detected; further, the user's head and/or neck can also be detected. , and the corresponding sensors are also set separately, so that the motion data of different parts of the user's body can be detected more accurately.

[0047] After detecting the above action data by the sensor, the system directly acquires the above action data detected by the sensor.

[0048] Step S20: Parse the acquired action data according to the set position of the sensor on the user's body, and obtain an action currently performed by the user;

[0049] After acquiring the motion data detected by the sensors respectively set at different positions on the user, the system sets the position according to the sensor on the user's body, for example: the left and right hands, the left and right feet, the waist or other parts of the sensor, the analysis Obtaining corresponding actions of the sensors of the different setting positions mentioned above Data, so that the action currently performed by the user can be known.

[0050] In the embodiment of the present invention, a sensor is respectively disposed on the hands, the feet, and the waist of the user as an example, and an example is illustrated. For example, the sensor that is separately set by the user's hands, feet, and waist is regarded as five points at different setting positions. When the user stands normally, the original position of the above five points is acquired; when the user is upside down, Obtaining the action data of the above five points in the user's inverted position, by analyzing the above action data, obtaining the new position of the above five points after the user's motion, comparing the new position with the original position, and obtaining the current user The action performed is an inverted action.

[0051] Step S30: The action currently performed by the obtained user is fed back to the corresponding virtual character in the virtual reality, and the corresponding virtual character in the virtual reality is controlled to perform a corresponding operation.

[0052] After the system parses and obtains the action currently performed by the user, the action currently performed by the obtained user is fed back to the corresponding virtual character in the virtual reality, for example: the system converts the action performed by the user into virtual reality. Corresponding virtual characters can recognize the control commands, thereby controlling the corresponding virtual characters in the virtual reality to perform corresponding operations.

[0053] For example, if the user performs the action of "right hook", the system retrieves the corresponding sensor data and parses out the "right hook action" performed by the user, thereby feeding back the "right hook action" to The corresponding virtual character A in the virtual reality controls the virtual character A to perform the same "right hooking action" as the user.

[0054] In a preferred embodiment, as shown in FIG. 2, in the virtual reality-based user interaction method of the present invention, in the embodiment of FIG. 1, the above-mentioned "Step S30, the action feedback of the currently performed user to be acquired" To the corresponding virtual character in the virtual reality, and controlling the corresponding virtual character in the virtual reality to perform the corresponding operation" may be implemented as steps S301-S302 as described below:

[0055] Step S301: According to the currently performed action of the acquired user, draw a motion track corresponding to the action currently performed by the user, and obtain motion track information after the drawing;

In the embodiment of the present invention, after the system parses the motion data detected by the sensor and obtains the action currently performed by the user, according to the action currently performed by the acquired user, the motion track corresponding to the action currently performed by the user is drawn. Thereby, the motion track information after drawing is obtained.

[0057] Further, in a preferred embodiment, the motion trajectory information drawn by the system can be recognized by the virtual character in the virtual reality and directly perform the action corresponding to the motion trajectory information.

[0058] Step S302: Control the corresponding virtual character in the virtual reality according to the drawn motion track information. Corresponding operations are performed in accordance with the motion trajectory information.

[0059] According to the motion track information after the drawing, the system directly controls the corresponding virtual character in the virtual reality to perform the corresponding operation according to the motion track information. The specific implementation manner includes: but is not limited to: the system directly sends the motion track information to a corresponding virtual character, and the virtual character identifies an operation corresponding to the motion track information, and performs an operation corresponding to the motion track information. Alternatively, the system controls the corresponding virtual character to perform an operation consistent with the action corresponding to the motion track information according to the motion track information.

[0060] In a preferred embodiment, based on the description of the embodiment shown in FIG. 1 and FIG. 2, as shown in FIG. 3, the virtual reality-based user interaction method of the present invention further includes the following steps:

[0061] Step S40: Obtain environment information of a scenario in which the virtual character is currently located in the virtual reality, and feed the environment information to the user, so that the user can perceive the environment information.

In the embodiment of the present invention, the system may also obtain the environment information corresponding to the scenario in which the virtual character is currently located in the virtual reality, and feed the environment information to the user, so that the user can perceive the environment information, and then Corresponding tactile perception. The environment information corresponding to the scene in which the virtual character is currently located in the virtual reality includes, but is not limited to, temperature information, humidity information, air pressure information, wind direction information, and rainfall or snowfall information.

[0063] For example, the system obtains the scene in which the virtual character is currently located in the virtual reality: the virtual character B is in the hail, and the environment information corresponding to the scene “ice” is: the temperature is minus 10° C. The system changes the current environment of the user according to the above environment information, and then feeds the environment information to the user, so that the user can perceive the environment information through the environment in which the user is currently located. For example, the user also feels cold.

[0064] In the embodiment of the present invention, the system changes the specific change mode of the environment in which the user is currently located, according to the specific environment in which the user is located; for example, if the user is currently in a closed space, the system passes Control the smart home in the space, such as air conditioning, to adjust the user's current environment. For another example, if the user currently wears a garment having adjustable environmental factors (such as temperature, humidity, etc.) to perform the above actions, the system adjusts the environmental factors (such as temperature and humidity) of the functional clothing. The user perceives the above environmental information.

[0065] The user interaction method based on virtual reality of the present invention obtains the transmission of different positions on the user's body by acquiring The action data detected by the sensor; the action data obtained by the user is obtained according to the set position of the sensor on the user's body, and the action currently performed by the user is obtained; and the action currently performed by the user is fed back to the virtual reality. a virtual character, and controlling the corresponding virtual character in the virtual reality to perform a corresponding operation; having a sensor worn by the user to detect and acquire the user's physical motion, thereby enabling the user to interact with the virtual character in the virtual scene The beneficial effect is to improve the realism of the virtual reality; further, since the user can also perceive the environmental information of the virtual character, the user experience is also improved.

[0066] A virtual reality-based user interaction system method corresponding to the embodiment described in FIG. 1, FIG. 2 and FIG. 3, the present invention also provides a virtual reality-based user interaction system, as shown in FIG. The virtual reality-based user interaction system includes: a plurality of sensors 100 that are worn at different locations on the user's body, and the virtual reality-based user interaction system further includes: an acquisition module 60, a parsing module 70, and a control module 80. ; among them:

[0067] The obtaining module 60 is configured to acquire motion data detected by the sensor 100 worn at different positions on the user's body;

In the embodiment of the present invention, the user wears the corresponding sensor 100 in different positions on the body in advance. When the user moves, the sensor 100 located in different parts of the user can know the user through the detected corresponding data. The corresponding action made.

[0069] In a preferred embodiment, the sensor 100 worn at different locations on the user's body comprises: sensors respectively disposed on the user's hands, feet and waist. For example, respective sensors are provided on the user's hands, feet, and waist, respectively, so that motion data of the user's hands, feet, and body torso portions can be detected; further, the user's head and/or neck can also be detected. The corresponding sensor 100 is also set separately, so that the motion data of different parts of the user's body is detected more accurately.

[0070] After the motion data is detected by the sensor 100, the acquisition module 60 directly acquires the motion data detected by the sensor 100.

[0071] The parsing module 70 is configured to parse the obtained action data according to the set position of the sensor 100 on the user's body, and obtain an action currently performed by the user;

[0072] After the obtaining module 60 obtains the action data detected by the sensor 100 respectively set in different positions on the user, the parsing module 70 sets different positions according to the sensor 100 on the user's body, for example: The sensor 100 of the left and right hands, the left and right feet, the waist, or other parts analyzes the motion data corresponding to the sensors 100 of the different set positions obtained, so that the current action performed by the user can be known.

[0073] In the embodiment of the present invention, a sensor is respectively disposed on the hands, the feet, and the waist of the user as an example, and an example is illustrated. For example, the sensor respectively set by the user's hands, feet and waist is regarded as five points located at different setting positions, and when the user stands normally, the obtaining module 60 acquires the original positions of the above five points; When the handstand is inverted, the obtaining module 60 acquires the action data of the five points in the user's inverted position, and the parsing module 70 obtains the new position of the five points after the user moves by analyzing the action data. The new position is compared with the original position, and the current action performed by the user is obtained as an inverted action.

[0074] The control module 80 is configured to feed back the currently performed action of the acquired user to a corresponding virtual character in the virtual reality, and control the corresponding virtual character in the virtual reality to perform a corresponding operation.

[0075] After the parsing module 70 parses and obtains the action currently performed by the user, the control module 80 feeds back the action currently performed by the obtained user to the corresponding virtual character in the virtual reality, for example, the control module 80 executes the user. The action is converted into a control command that can be recognized by the corresponding virtual character in the virtual reality, thereby controlling the corresponding virtual character in the virtual reality to perform a corresponding operation.

[0076] For example, if the user performs the action of "right hook", the acquisition module 60 obtains the corresponding sensor data, and the parsing module 70 parses the "right hook action" performed by the user, and the control module 80 thereby "" The right hook action "feeds back to the corresponding virtual character A in the virtual reality, and controls the virtual character A to perform the same "right hook action" as the user.

[0077] In a preferred embodiment, as shown in FIG. 5, in the virtual reality-based user interaction system of the present invention, in the embodiment of FIG. 1, the control module 80 includes: a drawing unit 801 and a control unit 802. ; among them:

[0078] The drawing unit 801 is configured to: according to the obtained action currently performed by the user, draw a motion track corresponding to the action currently performed by the user, and obtain motion track information after the drawing;

In the embodiment of the present invention, after the parsing module 70 parses the action data detected by the sensor 100 and obtains the action currently performed by the user, the drawing unit 801 draws the current execution of the user according to the obtained action currently performed by the user. The motion trajectory corresponding to the motion, so as to obtain the motion trajectory information after the drawing [0080] Further, in a preferred embodiment, the motion trajectory information drawn by the rendering unit 801 can be recognized by the virtual character in the virtual reality and directly perform the action corresponding to the motion trajectory information.

[0081] The control unit 802 is configured to control, according to the drawn motion trajectory information, a corresponding virtual character in the virtual reality to perform a corresponding operation according to the motion trajectory information.

[0082] According to the above-mentioned motion trajectory information drawn by the rendering unit 801, the control unit 802 directly controls the corresponding virtual character in the virtual reality to perform the corresponding operation according to the motion trajectory information. The specific implementation manner includes, but is not limited to, the control unit 802 directly transmitting the motion track information to the corresponding virtual character, and the virtual character identifies the operation corresponding to the motion track information, and performs the operation corresponding to the motion track information. Alternatively, the control unit 802 controls the corresponding virtual character to perform an operation consistent with the action corresponding to the motion track information according to the motion track information.

[0083] In a preferred embodiment, based on the description of the embodiment shown in FIG. 4 and FIG. 5, as shown in FIG. 6, the virtual reality-based user interaction system of the present invention further includes:

[0084] The interaction module 90 is configured to obtain environment information of a scenario in which the virtual character is currently located in the virtual reality, and feed the environment information to the user, so that the user can perceive the environment information.

In the embodiment of the present invention, the interaction module 90 can also obtain the environment information corresponding to the scene in which the virtual character is currently located in the virtual reality, and feed the environment information to the user, so that the user can perceive the environment information, and then Corresponding tactile perception. The environment information corresponding to the scenario in which the virtual character is currently located in the virtual reality includes, but is not limited to, temperature information, humidity information, air pressure information, wind direction information, and rainfall or snowfall information.

[0086] For example, the interaction module 90 obtains the scene in which the virtual character is currently located in the virtual reality: the virtual character B is in the hail, and the environment information corresponding to the scene "ice" is: the temperature is minus 10 ° C The interaction module 90 changes the environment in which the user is currently located according to the environment information, and then feeds the environment information to the user, so that the user can perceive the environment information through the environment in which the user is currently located. For example, the user also feels To the cold.

In the embodiment of the present invention, the interaction module 90 changes the specific change manner of the environment in which the user is currently located, and is executed according to the specific environment in which the user is located; for example, if the user is currently in a closed space, The interaction module 90 adjusts the environment in which the user is currently located by controlling a smart home such as an air conditioner in the space. For another example, if the user is currently wearing an adjustable environmental factor (such as temperature, humidity, etc.) When the garment performs the above actions, the interaction module 90 causes the user to perceive the environmental information by adjusting environmental factors (such as temperature and humidity) of the functional garment.

[0088] The virtual reality-based user interaction system acquires action data detected by sensors worn at different positions on the user's body; and according to the set position of the sensor on the user's body, the acquired action data is parsed, Obtaining an action currently performed by the user; feeding back the currently performed action of the obtained user to a corresponding virtual character in the virtual reality, and controlling the corresponding virtual character in the virtual reality to perform a corresponding operation; having detecting by using a sensor worn by the user And obtaining the user's physical movements, thereby enabling the user to interact with the virtual characters in the virtual scene, thereby improving the realism of the virtual reality; further, since the user can also perceive the environmental information of the virtual character, It also improves the user experience.

[0089] It is to be understood that the terms "comprising", "comprising", or any other variants are intended to encompass a non-exclusive inclusion, such that a process, method, article, or device comprising a Those elements, but also other elements not explicitly listed, or elements that are inherent to such a process, method, item or device. An element defined by the phrase "comprises a ..." without further restrictions does not exclude the presence of additional elements in the process, method, article, or device that comprises the element.

[0090] The foregoing serial numbers of the embodiments of the present invention are merely for the description, and do not represent the advantages and disadvantages of the embodiments.

Through the description of the above embodiments, those skilled in the art can clearly understand that the foregoing embodiment method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be through hardware, but in many cases, the former It is a better implementation. Based on such understanding, the technical solution of the present invention, which is essential or contributes to the prior art, may be embodied in the form of a software product stored in a storage medium (such as ROM/RAM, disk, The optical disc includes a number of instructions for causing a terminal device (which may be a mobile phone, a computer, a server, or a network device, etc.) to perform the methods described in various embodiments of the present invention.

The above description is only a preferred embodiment of the present invention, and is not intended to limit the scope of the patents. The equivalent structure or equivalent process transformations made by the description of the present invention and the drawings are directly or indirectly applied to other related technical fields. The same is included in the scope of patent protection of the present invention.

Claims

Claim

A virtual reality-based user interaction method, comprising the steps of: acquiring motion data detected by sensors worn at different positions on a user's body; and parsing and obtaining according to the set position of the sensor on the user's body The action data acquires an action currently performed by the user;

The action currently performed by the obtained user is fed back to the corresponding virtual character in the virtual reality, and the corresponding virtual character in the virtual reality is controlled to perform a corresponding operation.

The method of claim 1, wherein the virtual reality based user interaction method further comprises:

The environment information of the scenario in which the virtual character is currently located in the virtual reality is obtained, and the environment information is fed back to the user, so that the user can perceive the environment information.

The method according to claim 2, wherein the environmental information comprises: temperature information, humidity information, air pressure information, wind direction information, and rainfall or snowfall amount.

The method of claim 1, wherein the sensors worn at different locations on the user's body comprise:

Sensors are placed separately on the user's hands, feet and waist.

The method according to any one of claims 1 to 4, wherein the action currently performed by the acquired user is fed back to a corresponding virtual character in the virtual reality, and the corresponding virtual character in the virtual reality is controlled to be executed. The corresponding operations, including:

According to the currently performed action of the obtained user, the motion track corresponding to the action currently performed by the user is drawn, and the motion track information after the drawing is obtained;

The corresponding virtual character in the virtual reality is controlled to perform a corresponding operation according to the motion track information according to the drawn motion track information.

A virtual reality based user interaction system, the system comprising a plurality of sensors worn at different locations on a user's body;

The VR-based user interaction system further includes:

An acquisition module for acquiring sensors detected by sensors worn at different positions on the user's body Action data

The parsing module is configured to parse the acquired action data according to the set position of the sensor on the user's body, and obtain an action currently performed by the user;

a control module, configured to feed back the currently performed action of the acquired user to a corresponding virtual character in the virtual reality, and control the corresponding virtual character in the virtual reality to perform a corresponding operation. The system according to claim 6, wherein the system is characterized in that: The virtual reality based user interaction system further includes:

The interaction module is configured to obtain environment information of a scenario in which the virtual character is currently located in the virtual reality, and feed the environment information to the user, so that the user can perceive the environment information.

The system according to claim 7, wherein the environmental information comprises: temperature information, humidity information, air pressure information, wind direction information, and rainfall or snowfall amount.

The system of claim 6 wherein said sensors worn at different locations on a user's body comprise:

Sensors are placed separately on the user's hands, feet and waist.

The system according to any one of claims 6 to 9, wherein the control module includes a drawing unit, configured to draw a motion track corresponding to the currently performed action of the user according to the obtained action currently performed by the user, The motion track information after drawing;

And a control unit, configured to control, according to the drawn motion trajectory information, a corresponding virtual character in the virtual reality to perform a corresponding operation according to the motion trajectory information.