WO2014067058A1

WO2014067058A1 - Interface switching method and apparatus

Info

Publication number: WO2014067058A1
Application number: PCT/CN2012/083721
Authority: WO
Inventors: 宣曼; 黄晨; 薛传颂
Original assignee: 华为技术有限公司
Priority date: 2012-10-30
Filing date: 2012-10-30
Publication date: 2014-05-08
Also published as: CN103180803A; CN103180803B

Abstract

Embodiments of the present invention provide an interface switching method and apparatus. The method comprises: after detecting user information, identifying a first posture of a user from the user information; if the first posture of the user is an interface switching posture, displaying prompt information in a specified time period, wherein the prompt information is used to prompt the user to input a second posture; when detecting user information in the specified time period, identifying the second posture of the user; if the second posture of the user is a switching acknowledgement posture, executing an interface switching operation associated with the first posture. According to the present invention, defects of a high misjudgment rate or long waiting time for identification of an interface switching posture instruction in a somatosensory interaction scenario can be solved, accuracy of posture control is raised and user experience is improved.

Description

Method and device for interface switching

Technical field

The present invention relates to the field of communication network technologies, and in particular, to a method and apparatus for interface switching in a somatosensory interaction scenario. Background technique

Vision-based somatosensory interaction means that the computer captures the user's image through the camera, and uses pattern recognition, artificial intelligence and other technologies to understand the meaning of the user's actions, providing a more natural and intuitive way of interacting with the body. Currently widely used in augmented reality, somatosensory game control and other scenarios. In the process of somatosensory interaction, the somatosensory interactive application system captures a video frame containing user information through a camera, and then obtains information of the user in the video frame (for example, joint point information) through image analysis technology, thereby determining the posture of the user (pose) and A gesture composed of a posture change in a continuous video frame; a gesture and a motion of the user together constitute a gesture, and the somatosensory interactive application system performs a corresponding feedback operation according to an instruction corresponding to the posture of the user. This constitutes a complete vision-based somatosensory interaction process. In the prior art, for the judgment of the interface switching posture, the manner of capturing is: firstly, the posture input by the user is recognized, and when the predetermined interface switching posture is satisfied, the user is required to maintain the posture for a period of time before the interface switching instruction is triggered. For example, when using the Microsoft's somatosensory game device Kinect for a somatosensory game, the user can exit the game by "stretching the left arm and tilting it 45° to the body", and asking the user to keep the gesture for a while to trigger the exit. Game "Operation. Otherwise cancel the operation and retain the original game interface. If the waiting time is set to a short time, it is easy to misjudge some of the user's unintended operations as an interface switching instruction. If the waiting time is set longer, the user needs to keep a certain posture for a long time, and the user experience is poor. Therefore, the prior art has a problem that the false positive rate or the waiting time is long when the interface switching instruction is executed. Summary of the invention The embodiment of the invention provides a method and a device for switching interfaces, which are used for improving the recognition accuracy of the interface switching posture instruction in the somatosensory interaction scenario and improving the user experience.

The first aspect, the method for interface switching provided by the embodiment of the present invention includes:

After detecting the user information, the first posture of the user is recognized from the user information; if the first posture of the user is an interface switching posture, prompt information is displayed within a specified time, and the prompt information is used to prompt the user to input the second Posture

When the user information is detected within the specified time, the second gesture of the user is recognized; if the second gesture of the user is the confirmation switching gesture, the interface switching operation associated with the first gesture is performed.

In a first possible implementation manner of the first aspect, after the identifying the second gesture of the user, the method further includes: if the second posture of the user is a cancel switching gesture, canceling the interface switching operation associated with the first gesture; or The user second gesture is not to confirm the switching posture or cancel the switching posture, continue to detect the user information, and returns to the step of recognizing the second posture of the user when the user information is detected within the specified time.

With reference to the first aspect or the first possible implementation manner of the first aspect, in a second possible implementation manner, after prompting the user to input the second gesture, the method further includes: when the user is not detected within the specified time In the case of information, the interface switching operation associated with the first gesture is cancelled.

In combination with the first aspect or the first possible implementation of the first aspect or the second possible implementation of the first aspect, in a third possible implementation, the identifying the first posture or the second posture of the user includes: Obtaining a joint point involved in the set posture; reading data of the joint point from the detected user information, wherein the user information includes user bone frame information, and the skeleton frame information includes a joint point Information and time stamp information; calculating a matching parameter value of the set posture according to the data of the joint point; and identifying the first posture or the second posture of the user according to the matching parameter value.

In conjunction with the third possible implementation of the first aspect, in a fourth possible implementation, the joint points involved in obtaining the set posture include: determining the current interface type and the current interface a set posture, obtaining a joint point involved in the posture set under the current interface; calculating a matching parameter value of the set posture according to the data of the joint point includes: calculating according to data of the joint point The matching parameter value of the posture set under the current interface.

In conjunction with the third possible implementation of the first aspect, in a fifth possible implementation, the joint point involved in obtaining the set posture includes: determining a default posture of the somatosensory interactive game application system, obtaining the default posture The joint point involved; calculating the matching parameter value of the set posture according to the data of the joint point comprises: calculating a matching parameter value of the default posture according to the data of the joint point.

With reference to the third possible implementation manner of the first aspect, in a sixth possible implementation manner, when the set posture is an action gesture, the reading the joint point from the detected user information The data includes: reading, according to a plurality of consecutive user skeleton frame information, joint point data corresponding to the joint point and time stamp information of the user bone frame; and calculating the data according to the joint point The matching parameter value of the set posture includes: calculating a displacement of the joint point based on the joint point data and the time stamp information.

With reference to the third possible implementation manner of the first aspect, in a seventh possible implementation manner, when the set posture is a posture posture posture, the reading the joint from the detected user information The data of the point includes: reading joint point data corresponding to the joint point involved in the set posture from the user skeleton frame information; and calculating a matching parameter value of the set posture according to the data of the joint point The method includes: calculating an angle of a bone between the joint points according to the joint point data.

With reference to the fourth possible implementation of the first aspect or the fifth possible implementation manner of the first aspect, in an eighth possible implementation, the first posture or the user is identified according to the matching parameter value The second posture includes: comparing the matching parameter value with a matching condition of the posture set under the current interface, or comparing the matching parameter value with a matching condition of a default posture of the somatosensory interaction application system; The posture corresponding to the matching parameter value matching the matching condition, the determined posture is the user first posture or the second posture.

In a second aspect, the apparatus for interface switching provided by the embodiment of the present invention includes: a detecting unit, configured to detect user information;

a first identifying unit, configured to: after the detecting unit detects the user information, identify the first posture of the user from the user information;

a display unit, configured to display prompt information within a specified time when the first gesture of the user is an interface switching gesture, where the prompt information is used to prompt the user to input a second gesture;

a second identifying unit, configured to: when the detecting unit detects the user information within the specified time, identify the second posture of the user;

The interface switching processing unit is configured to perform an interface switching operation of the first gesture association when the second gesture of the user is a confirmation switching gesture.

In a first possible implementation manner of the second aspect, the interface switching processing unit is further configured to cancel the interface switching operation associated with the first gesture when the second posture of the user is a cancel switching gesture.

With reference to the second aspect or the first possible implementation manner of the second aspect, in a second possible implementation manner, the interface switching processing unit is further configured to: when the user information is not detected within the specified time, The interface switching operation associated with the first gesture is cancelled.

With reference to the second aspect or the first possible implementation manner of the second aspect, or the second possible implementation manner of the second aspect, in a third possible implementation manner, the first identification unit or the second identification unit Contains:

a module for obtaining a joint point involved in the set posture; a reading module, configured to read data of the joint point from the detected user information, wherein the user information includes a user bone Frame information, the skeleton frame information includes joint point information and time stamp information; and a calculation module, configured to calculate a matching parameter value of the set posture according to the data of the joint point;

And an identification module, configured to identify the first posture or the second posture of the user according to the matching parameter value.

In conjunction with the third possible implementation of the second aspect, in a fourth possible implementation, Obtaining a module, further configured to obtain a joint point involved in the posture set in the current interface; and a calculating module, configured to calculate a matching parameter value of the posture set in the current interface according to the data of the joint point .

In conjunction with the third possible implementation of the second aspect, in a fifth possible implementation, the obtaining module is further configured to obtain a joint point involved in a default posture of the somatosensory interactive application system; the calculating module is further used for A matching parameter value of the default posture is calculated based on the data of the joint point.

In conjunction with the third possible implementation of the second aspect, in a sixth possible implementation, the reading module is further configured to: when the set posture is an action posture, from a plurality of consecutive user bones Reading, in the frame information, the joint point data corresponding to the joint point and the time stamp information of the user skeleton frame; the calculation module, further configured to calculate the joint point data and the time stamp information according to the joint point data Off the displacement of the node.

With reference to the third possible implementation manner of the second aspect, in a seventh possible implementation, the reading module is further configured to: when the set posture is an attitude posture, information from the user skeleton frame The joint point data corresponding to the joint point involved in the set posture is read; the calculation module is further configured to calculate a bone angle between the joint points according to the joint point data.

With reference to the fourth possible implementation of the second aspect, or the fifth possible implementation of the second aspect, in an eighth possible implementation, the identifying module is further configured to use the matching parameter value Comparing the matching conditions of the postures set under the current interface, or comparing the matching parameter values with the matching conditions of the default postures of the somatosensory interaction application system; determining matching parameter values that match the matching conditions The corresponding posture is a first posture or a second posture of the user.

It can be seen from the above technical solution that the mechanism for confirming the first posture instruction by using the second posture is effectively solved by the embodiment of the present invention, thereby effectively solving the problem that the interface switching posture recognition time is long or the false positive rate is high under the somatosensory interaction scene. , which improves the accuracy of posture control, which greatly enhances the user experience. BRIEF DESCRIPTION OF THE DRAWINGS In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings to be used in the embodiments or the description of the prior art will be briefly described below, and obviously, in the following description The drawings are only some of the embodiments of the present invention, and other drawings may be obtained from those skilled in the art without departing from the drawings. 1 is a flowchart of a method for switching an interface according to an embodiment of the present invention; FIG. 2 is a flowchart of a method for identifying a first posture or a second posture of a user according to an embodiment of the present invention; and FIGS. 3A-3D illustrate a method according to the present invention. The embodiment shows a graphical user interface of the device at different time points of the interface switching. FIG. 4 is a block diagram of the interface switching device according to an embodiment of the present invention; FIG. 5 is a schematic diagram of the interface switching device according to another embodiment of the present invention; FIG. 6 is a structural diagram of a computer system-based interface switching apparatus according to still another embodiment of the present invention. detailed description

The technical solutions in the embodiments of the present invention are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present invention. It is obvious that the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention. An embodiment of the present invention provides a method for interface switching, which can be used for interface switching in a somatosensory interaction scenario. The specific interface operation includes one of the following: exiting the application, returning to the upper-level interface, returning to the main interface, and the recruiting menu. The operation that causes the interface to change. Referring to Figure 1, the method includes: Step 101: After detecting the user information, identify the first posture of the user from the user information. The method for detecting the user information may be: acquiring the skeleton frame information of the user, and determining whether the acquired skeleton frame information of the user includes valid joint point data. If valid joint point data is included, the user information is detected. Otherwise, the user information is not detected and the test is continued.

The method for recognizing the first posture of the user may be: obtaining a joint point involved in the preset posture; and reading valid joint point data corresponding to the joint point involved in the preset posture from the detected user information; The effective joint point data calculates a matching parameter value that matches the preset posture; and then the user's first posture is identified based on the matching parameter value. The first posture of the user may be an action posture or a posture posture.

Specifically, in an implementation manner, the somatosensory interactive application system is used as an execution subject, and the user's skeleton frame information is obtained from the somatosensory interaction device (in this embodiment, the somatosensory game product Kinect). Specifically, the SDK (Software Development Kit) provided by the Kinect device of the somatosensory game product includes a skeleton frame information extraction function NuiSkeletonGetNextFrame, and the application can extract the user skeleton frame information of the current time from the kinect device by calling the function, regardless of At present, there is no user in front of the Kinect device, and the device will generate a frame of user skeleton frame information.

The user skeleton frame information is represented by a NUI-SKELETON-FRAME data structure, and the data structure includes joint point information (using NUI-SKELETON-DATA data structure representation) and timestamp information (represented by liTimestamp parameter), joint The point information contains a judgment flag eTrackingState having valid joint point data. If the eTrackingState parameter is true, the user information is detected. Otherwise, the parameter value is false, indicating that no user information is detected and the detection continues.

Step 102: If the first gesture of the user is an interface switching gesture, the prompt information is displayed within a specified time, and the prompt information is used to prompt the user to input the second gesture.

Optionally, step 102 may further include the following steps: when the first posture of the user is not a boundary When the face switching gesture, the user information continues to be detected or when the user first gesture is not an interface switching gesture and an operation associated with the first gesture is set, the first gesture associated operation may also be performed.

The prompt information may include a second gesture selection item and an operation indication of the second posture. Specifically, the display manner of the prompt information may be a text, a picture, or the like, and a display effect such as blinking, fading, and the like may also be used. For example, the second posture selection item may be two options of “confirm switching posture” and “cancel switching posture”, and may be displayed by using a text box or a text, and correspondingly, the operation instruction of the second posture is “confirming the switching posture”. The operation method indication of ", "Cancel switching posture" can indicate how the user operates by using text, symbols, pictures or animations. The second posture may be an action posture or a posture posture. Step 103: Identify the user's second gesture when the user information is detected within the specified time. The method of detecting the user information is the same as the method of detecting the user information in step 101, and the method of recognizing the second posture of the user is the same as the method of recognizing the first posture of the user in step 101. The step 103 may further include ignoring the interface switching operation associated with the first gesture if the user information is not detected within the specified time. Step 104: If the second posture of the user is a confirmation switching posture, perform an interface switching operation of the first posture association. Further, if the second gesture is a cancel switching gesture, if yes, the interface switching operation associated with the first gesture is ignored. Alternatively, if the second gesture is neither a confirmation switching gesture nor a cancel switching gesture, returning to step 103, the user information is continuously detected for a specified time. An embodiment of the present invention provides a method for recognizing a first posture of a user after detecting user information in a method of interface switching, and a method for recognizing the second gesture is similar to the method, and details are not described herein. Referring to Figure 2, the method includes:

Step 201: Obtain a joint point involved in the set posture;

The step specifically includes: determining a current interface type and a posture set under the current interface. Potential, obtain the joint points involved in setting the posture under the current interface.

Alternatively, the step may specifically include: determining all the default gestures of the somatosensory interaction application system, and obtaining joint points involved in all the default gestures of the system.

Specifically, in an embodiment, the state of the state machine is switched through the interface, and the current interface is determined to be an application interface before the switching, and the posture set under the current interface is only the interface switching posture; the interface switching posture is a shape of the left arm 45. Degree" posture, there are 7 joint points involved: SHOULDER - CENTER (central shoulder joint point), SHOULDER - RIGHT (right shoulder joint point) and ELBOW- RIGHT (right elbow joint point;), WRIST RIGHK right wrist joint point; ), SHOULDER—LEFT (left shoulder joint point), ELBOW—LEFT (left elbow joint point), WRIST LEFT (left wrist joint point;).

Alternatively, the state of the state machine is switched through the interface, and the current interface is determined to be a switching prompt interface, and the posture set under the current interface has a confirmation switching posture and a cancel switching posture; wherein, the confirmation switching posture is a left-handed right-wing motion The posture, the joint point involved is HAND-LEFT (left hand joint point), the cancel switching posture is a left-handed left-wing action posture, and the joint points involved are HAND-LEFT (left-hand joint point).

Step 202: Read data of the joint point from the detected user information; wherein the user information includes user skeleton frame information, and the user skeleton frame information includes joint point information and time stamp information.

When the set posture is the posture posture, the step 202 may specifically include: specifically reading the joint point data corresponding to the joint point involved in the set posture from the user skeleton frame information.

When the gesture is a gesture, the step 202 may include: reading, from the plurality of consecutive user skeleton frame information, the joint point data of the joint point and the time of the user skeleton frame related to the set posture. Stamp information.

When there is the same joint point among the multiple joint points involved in the set posture, the joint point data only needs to be read once for each frame of bone information.

In a specific implementation, for example, the posture set under the current interface is an interface switching posture. Because the posture is a posture posture, only the coordinate data of the seven joint points involved in the posture in the current skeleton frame information is read, such as a table. 1 is shown.

Joint Joint Posit

Index! Joint 1011 (joint Point number ') point position)

a n L U LU Pv ϊ s \ I Xi' yi' Zi

Heart shoulder joint point)

b SHOULDER _m RIGHT (right shoulder ^χ 2· Υΐ· ^ζ 2

Joint point)

c ELBOW _ RIGHT (right elbow close ^Χ 3, ^Υ 3, ^z 3

τ ^ )

d WRIST _ RIGHT (right wrist off ^4> y ₄ < _3⁄4

τ )

e SHOULDER-LEFT (left shoulder

Joint point

f ELBOW—LEFT (left, hour joint ^χ 6· Υβ· ^z 6

Point :)

g WRIST _ LEFT (left wrist joint

Point)

For example, for the posture set under the current interface, there are a confirmation switching posture and a cancellation switching posture, and the two postures involve the same joint point (left hand joint point), so the joint point data only needs to be read once for each frame of bone information. . Because the two postures are action postures, it is necessary to continuously read the left hand joint point coordinate data of the user's multiple bone frames and the timestamp information of each frame, and the current detection timestamp is the start time stamp ^, as shown in Table 2. .

Step 203: Calculate a matching parameter value of the set posture according to the data of the joint point.

The matching parameter value for calculating the set posture includes: calculating a matching parameter value of all the default postures of the posture set under the current interface or the somatosensory interaction application system. Wherein, when the same matching parameter exists in the set posture, only the value of the matching parameter needs to be calculated once.

For example, for the posture set under the current interface is the interface switching posture, the matching parameter is 4 bone angles, taking Zabc as an example, which is composed of the central shoulder joint point a and the right shoulder joint point joint point b in Table 1. The angle between the bone and the bone of the right shoulder joint point b and the right elbow joint point c. The angle is calculated as:

Ac = cos 1, ac ² ^ ( i -- x ₃ ) ² + (y _t - y ₃ ) ² ; ah (x ₁ ― x ₂ Y + (y ₁ ― y ₂ Y\ be ² = (x ₂ ― x ₃ ) ² ( ₂ - y ₃ ) ² ,

With a similar formula, you can calculate the parameter values of the other three bone angles (Zbcd, aef, Zefg). For example, for the posture set under the current interface, there is a confirmation switching posture and a cancel switching posture, and the matching parameter for confirming the switching posture is the displacement of the left hand joint point, and the matching parameter for canceling the switching posture is also the displacement of the left hand joint point. There are the same matching parameters for both poses, so only the value of the match parameter is calculated once. Wherein, the displacement calculation formula of the left hand joint point is the total displacement s = ∑i _{= 2} As _k between the initial detection time and the end time ^ in Table 2, and the displacement between the adjacent two time stamps ^ is ^ = x _; - x _; _ ₁ Compare the positive and negative signs of the displacement ^ with the total displacement s. The same positive and negative signs indicate that the calculation needs to be continued. If the positive and negative signs are different, the gesture ends. Before the preset time is reached, the end of the gesture is detected, the timing and calculation are stopped, otherwise the time is counted and calculated until the preset time point, and the total displacement s is the displacement value of the left hand joint point.

Step 204: Identify the first posture of the user according to the matching parameter value.

Specifically, the step may be: comparing the matching parameter value with a matching condition of the posture set under the current interface, or comparing the matching parameter value with a matching condition of a default posture of the somatosensory interaction application system. Determining a posture corresponding to the matching parameter value that matches the matching condition, and determining the posture is the first posture of the user.

For example, for the posture set under the current interface is the interface switching posture, according to the calculated four bone angles, it is determined whether the matching condition Zabc = 135 is satisfied at the same time. ±10°, Zbcd=180. ±10. , Zaef=90. ±10. , Zefg=180. ±10. If yes, identify the user's posture The potential is the interface switching posture, otherwise the user gesture is not the interface switching posture; in order to avoid the influence of the unintended action of the user, the user information detected in consecutive frames can be recognized as the setting posture.

For example, if the posture set under the current interface is the confirmation switching posture and the cancel switching posture, according to the calculated displacement value of the left hand joint point, it is determined whether s>0.3m or s<-0.3m is satisfied, if s>0.3m The user gesture is identified as a confirmation switching gesture, and if s<-0.3m identifies the user gesture as canceling the switching gesture, and s is another range, identifying the user gesture is not confirming the switching posture or canceling the switching posture.

For a further understanding of the present invention, an embodiment is provided to illustrate the GUI display of the device at different points in time of interface switching. Please refer to Figure 3A-3D. Take the exit game as an example. The display interface at different time points is as follows:

Before the interface is switched, that is, before the game exit gesture is received, as shown in FIG. 3A, the device 300 includes a display screen 301, and the current game screen 302 is displayed on the display screen.

After displaying the interface 3A before switching, the somatosensory interactive application system recognizes that the user inputs the exiting game gesture, thereby displaying the prompt interface containing the prompt information within the specified time. A cueing interface is shown in Fig. 3B: includes a game screen 302 before exiting, and prompt information prompting the user to input the second gesture. The prompt information appears in the lower left corner of the game screen 302 in a superimposed manner, and includes a "confirm exit", a "cancel exit" second gesture prompt item 303, and a second gesture operation instruction 304 corresponding to the two prompt items. The second gesture prompt item is displayed in a text manner, and the text is blinking to remind the user of the attention, and the second gesture operation indication includes the words "wave left", "wave to the right", and left/right arrow graphic symbols for indicating The right swing manual is used as the confirmation exit gesture, and the confirmation can be triggered to exit the game operation, and the leftward swing is manually canceled as the cancel exit gesture, and the cancellation of the exit game operation can be triggered.

Another prompt interface is shown in FIG. 3C: In addition to the game screen 302 before exiting, and the prompt information prompting the user to input the second gesture, optionally, the timing progress disk 305 is included, and as time progresses, black The sector area is gradually reduced, indicating a decrease in the remaining time that allows the user to input the second gesture by the reduction of the black sector area. The prompt information includes a second gesture prompt item 306 and a second gesture operation indication 307, wherein the second gesture prompt item 306 includes "confirm exit", "cancel exit", and is displayed in a text box manner, and the corresponding second gesture operation indication 307 Contains the words "left hand lift", "left hand lift" and posture diagram, used to indicate the left hand gesture In order to confirm the exit posture, it is possible to trigger the confirmation to exit the game operation, and the left hand gesture is to cancel the exit gesture, and the cancel cancellation game operation can be triggered. Optionally, when entering the prompt interface, the prompt information appears in a fade-in manner, and when exiting the prompt interface, the prompt information disappears in a fade-out manner.

After the prompt interface 3B or 3C is displayed, if the somatosensory interactive application system receives the confirmation exit posture of the user within a specified time, and exits the game, the interface after the display exits is as shown in FIG. 3D: The game menu screen 308 is included.

After the prompt interface 3B or 3C is displayed, if the somatosensory interactive application system receives the user's cancel exit gesture within a specified time, or the somatosensory interactive application system neither receives the user's confirmation exit gesture nor receives the user's cancel exit gesture. , then show the interface 3A before switching. An embodiment of the present invention provides an apparatus for interface switching. Referring to FIG. 4, the interface switching apparatus 400 includes: a detecting unit 401, configured to detect user information;

The first identifying unit 402 is configured to: after the detecting unit detects the user information, identify the first posture of the user from the user information;

The display unit 403 is configured to display prompt information in a specified time when the first posture of the user is an interface switching posture, where the prompt information is used to prompt the user to input the second posture;

a second identifying unit 404, configured to: when the detecting unit detects user information within the specified time, identify a second posture of the user;

The interface switching processing unit 405 is configured to perform an interface switching operation associated with the first gesture when the second posture of the user is a confirmation switching posture.

Optionally, the interface switching processing unit 405 is further configured to cancel the interface switching operation associated with the first gesture when the second gesture of the user is to cancel the switching gesture.

Optionally, the interface switching processing unit 405 is further configured to cancel the interface switching operation associated with the first gesture when the user information is not detected within the specified time.

Referring to FIG. 5, in the interface switching device 400, the first identifying unit 402 may further include:

a first obtaining module 4021, configured to obtain a joint point involved in the set posture;

a first reading module 4022, configured to read the joint from the detected user information Point data, where the user information includes user skeleton frame information, and the skeleton frame information includes joint point information and time stamp information;

a first calculating module 4023, configured to calculate a matching parameter value of the set posture according to the data of the joint point;

The first identification module 4024 is configured to identify the first posture or the second posture of the user according to the matching parameter value;

Optionally, the first obtaining module 4021 is further configured to obtain a joint point involved in the posture set under the current interface;

Optionally, the first calculating module 4023 is further configured to calculate, according to the data of the joint point, a matching parameter value of the posture set in the current interface;

Optionally, the first obtaining module 4021 is further configured to obtain a joint point involved in a default posture of the somatosensory interactive application system;

Optionally, the first calculating module 4023 is further configured to calculate a matching parameter value of the default posture according to the data of the joint point;

Optionally, the first identification module 4024 is further configured to compare the matching parameter value with a matching condition of the posture set under the current interface, or interact the matching parameter value with the physical sense Comparing the matching conditions of the default posture of the application system; determining a posture corresponding to the matching parameter value that matches the matching condition, and determining the posture is the user first posture or the second posture;

Optionally, the first reading module 4022 is further configured to: when the set posture is an action posture, read a joint point related to the set posture from a plurality of consecutive user bone frame information. Corresponding joint point data and time stamp information of the user skeleton frame;

Optionally, the first calculating module 4023 is further configured to calculate the displacement of the joint point according to the joint point data and the timestamp information.

Optionally, the first reading module 4022 is further configured to: when the set posture is an attitude posture, read a joint point corresponding to the set posture from the user skeleton frame information. Off node data;

Optionally, the first calculating module 4023 is further configured to calculate a bone angle between the joint points according to the joint point data; Similar to the first identifying unit 402, the second identifying unit 404 may further include four modules: a second obtaining module 4041, a second reading module 4042, a second calculating module 4043, and a second identifying module 4044. The function of each module of the second identification unit 404 and the function of the corresponding module of the first identification unit 402 are also similar, and details are not described herein again.

The interface switching device in the embodiment of the present invention can be implemented based on a computer system, and the methods shown in FIG. 1 to FIG. 2 can be implemented in a computer system-based interface switching device. Figure 6 illustrates an embodiment of an interface switching device implemented in accordance with a computer system. The interface switching device in this embodiment may include: a processor 601, a memory 602, and a communication interface 603, where:

The communication interface 603 is configured to communicate with the somatosensory interaction device. Messages exchanged between the interface switching device and the somatosensory interaction device are transmitted and received through the communication interface 603. Specifically, the communication interface 603 is configured to acquire the skeleton frame information of the user from the somatosensory interaction device; the memory 602 is configured to store the program instructions; the processor 601 is configured to invoke the program instructions stored in the memory 602, and perform the following operations: after detecting the user information Identifying the user's first gesture from the user information; if the user's first gesture is an interface switching gesture, displaying prompt information within a specified time, the prompt information is used to prompt the user to input the second gesture; When the user information is detected within the specified time, the second posture of the user is recognized; if the second posture of the user is the confirmation switching posture, the interface switching operation associated with the first gesture is performed.

The processor 601 can be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or the like. The interface switching device in this embodiment may include a bus 604. The processor 601, the memory 602, and the communication interface 603 can be connected and communicated via the bus 604. The memory 602 may include: a random access memory (RAM), a read-only memory (ROM), a disk and the like having an storage function;

The processor 601 can also be used to perform the steps described in FIG. 1 to FIG. 2 in the method embodiment, and the embodiments of the present invention are not described in detail herein.

The foregoing provides a detailed description of the present invention. The principles and embodiments of the present invention are described herein with reference to specific examples. The description of the above embodiments is only to assist in understanding the method of the present invention and its core ideas; At the same time, for those skilled in the art, according to the idea of the present invention, there will be changes in the specific embodiments and application scopes. The description is not to be construed as limiting the invention.

Claims

Rights request

1. A method of interface switching, characterized in that the method includes:

After detecting the user information, identify the user's first gesture from the user information; if the user's first gesture is an interface switching gesture, display prompt information within a specified time, and the prompt information is used to prompt the user to enter the second posture;

When the user information is detected within the specified time, the user's second gesture is recognized; if the user's second gesture is a confirmation switching gesture, the interface switching operation associated with the first gesture is performed.

2. The method according to claim 1, wherein the identifying the second gesture of the user further includes:

If the user's second gesture is a switching gesture, cancel the interface switching operation associated with the first gesture; or

If the user's second gesture is not a confirmation switching gesture or a cancellation switching gesture, continue to detect the user information, and return to the step of identifying the user's second gesture when the user information is detected within the specified time.

3. The method according to claim 1 or 2, characterized in that, after prompting the user to input the second gesture, the method further includes:

When the user information is not detected within the specified time, the interface switching operation associated with the first gesture is canceled.

4. The method according to any one of claims 1 to 3, characterized in that the identifying the user's first gesture or the second gesture includes:

Obtain the joint points involved in the set pose;

Read the data of the joint points from the detected user information, where the user information includes user skeleton frame information, and the skeleton frame information includes joint point information and timestamp information; Calculate the matching parameter value of the set posture according to the data of the joint point;

The first gesture or the second gesture of the user is identified according to the matching parameter value.

5. The method according to claim 4, characterized in that the joint points involved in obtaining the set posture include:

Determine the current interface type and the posture set under the current interface, and obtain the joint points involved in the posture set under the current interface;

Calculating the matching parameter value of the set posture based on the data of the joint point includes: calculating the matching parameter value of the set posture under the current interface based on the data of the joint point.

6. The method according to claim 4, characterized in that the joint points involved in obtaining the set posture include:

Determine the default posture of the somatosensory interactive application system, and obtain the joint points involved in the default posture; Calculating the matching parameter value of the set posture based on the data of the joint points includes: Calculating the matching parameter values based on the data of the joint points Describes the matching parameter values for the default pose.

7. The method according to claim 5 or 6, characterized in that identifying the user's first gesture or second gesture according to the matching parameter value includes:

Compare the matching parameter value with the matching condition of the posture set under the current interface, or compare the matching parameter value with the matching condition of the default posture of the somatosensory interactive application system;

Determine the posture corresponding to the matching parameter value that matches the matching condition, and the determined posture is the first posture or the second posture of the user.

8. The method according to claim 4, characterized in that when the set posture is an action posture,

Reading the data of the joint points from the detected user information includes: reading the joint point phases involved in the set posture from multiple consecutive user skeleton frame information. The corresponding joint point data and the timestamp information of the user's skeleton frame;

Calculating the matching parameter value of the set posture according to the data of the joint point includes: calculating the displacement of the joint point according to the data of the joint point and the time stamp information.

9. The method according to claim 4, characterized in that when the set posture is a posture posture,

Reading the data of the joint points from the detected user information includes: reading the corresponding joint point data of the joint points involved in the set posture from the user skeleton frame information;

Calculating the matching parameter value of the set posture according to the data of the joint points includes: calculating the bone angle between the joint points according to the data of the joint points.

10. An interface switching device, characterized in that the device includes:

Detection unit, used to detect user information;

A first recognition unit, configured to recognize the user's first gesture from the user information after the detection unit detects the user information;

A display unit configured to display prompt information within a specified time when the user's first posture is an interface switching posture, and the prompt information is used to prompt the user to input the second posture;

The second recognition unit is configured to recognize the user's second gesture when the detection unit detects user information within the specified time;

An interface switching processing unit, configured to perform an interface switching operation associated with the first gesture when the user's second gesture is a confirmation switching gesture.

11. The device according to claim 10, characterized in that:

The interface switching processing unit is also configured to cancel the interface switching operation associated with the first gesture when the user's second gesture is a switching cancellation gesture.

12. The device according to claim 10 or 11, characterized in that:

The interface switching processing unit is also configured to when the user is not detected within the specified time When user information is obtained, the interface switching operation associated with the first gesture is canceled.

13. The device according to any one of claims 10 to 12, characterized in that the first identification unit or the second identification unit includes:

Obtain module, used to obtain the joint points involved in the set posture;

A reading module, configured to read the data of the joint points from the detected user information, where the user information includes user skeleton frame information, and the skeleton frame information includes joint point information and timestamp information. ;

A calculation module, used to calculate the matching parameter value of the set posture according to the data of the joint point;

An identification module, configured to identify the user's first posture or second posture according to the matching parameter value.

14. The device according to claim 13, characterized in that:

The obtaining module is further used to obtain the joint points involved in the posture set under the current interface;

The calculation module is further configured to calculate the matching parameter value of the posture set under the current interface according to the data of the joint points.

15. The device according to claim 13, characterized in that:

The obtaining module is further used to obtain the joint points involved in the default posture of the somatosensory interactive application system;

The calculation module is further configured to calculate the matching parameter value of the default posture according to the data of the joint point.

16. The device according to claim 14 or 15, characterized in that:

The identification module is further configured to compare the matching parameter value with the matching condition of the posture set under the current interface, or match the matching parameter value with the default posture of the somatosensory interactive application system. Compare the conditions; determine the matching parameters that match the matching conditions. The posture corresponding to the numerical value, the determined posture is the user's first posture or second posture.

17. The device according to claim 13, characterized in that:

The reading module is further configured to, when the set posture is an action posture, read the joint point data corresponding to the joint points involved in the set posture and the user's skeletal frame information from multiple consecutive user skeleton frame information. The timestamp information of the skeleton frame;

The calculation module is further configured to calculate the displacement of the joint point according to the joint point data and the timestamp information.

18. The device according to claim 13, characterized in that:

The reading module is further configured to read the joint point data corresponding to the joint points involved in the set posture from the user skeleton frame information when the set posture is a posture posture; the calculation A module further configured to calculate the bone angle between the joint points based on the joint point data.