CN116149491A - Interaction method, device, equipment and medium of wearable display equipment - Google Patents
Interaction method, device, equipment and medium of wearable display equipment Download PDFInfo
- Publication number
- CN116149491A CN116149491A CN202310166100.0A CN202310166100A CN116149491A CN 116149491 A CN116149491 A CN 116149491A CN 202310166100 A CN202310166100 A CN 202310166100A CN 116149491 A CN116149491 A CN 116149491A
- Authority
- CN
- China
- Prior art keywords
- rotation angle
- wearable display
- vector
- display device
- display interface
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
- G06F3/0346—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
- G06F3/038—Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The invention discloses an interaction method, device, equipment and medium of wearable display equipment, and is suitable for the technical field of wearable equipment. The method utilizes the initial vector and the gesture rotation angle acquired by the existing terminal equipment, and converts the initial vector and the gesture rotation angle into the final vector, namely, a virtual ray in the wearable display equipment is consistent with the rotation direction and the angle of the terminal equipment, and the operation of the wearable display equipment is realized through the control points existing between the virtual ray and the display interface of the wearable display equipment to replace the actual operation of the existing additional remote control device. The whole process does not need to add extra remote control equipment, only needs to be controlled through the existing terminal equipment of the user, saves cost, is convenient to carry, and improves the experience of the user.
Description
Technical Field
The invention relates to the technical field of wearable devices, in particular to an interaction method and device of a wearable display device, the wearable display device and a medium.
Background
In the actual use process of the wearable display equipment, the wearable display equipment can control the picture of the wearable display equipment to switch and rotate by adding an additional remote control device, and the additional remote control device is similar to a handle or a finger ring and other devices.
As users pursue weight reduction of wearable display apparatuses, the remote control device undoubtedly increases hardware support of the wearable display apparatuses, resulting in inconvenience in carrying. Meanwhile, the added extra remote control device increases the corresponding cost, and influences the experience of the user.
Therefore, a need exists for a person skilled in the art to improve the user experience without adding additional remote control devices.
Disclosure of Invention
The invention aims to provide an interaction method and device of a wearable display device, the wearable display device and a medium. The whole process does not need to add extra remote control equipment, only needs to be controlled through the existing terminal equipment of the user, saves cost, is convenient to carry, and improves the experience of the user.
In order to solve the technical problems, the invention provides an interaction method of a wearable display device, comprising the following steps:
acquiring an initial vector and a gesture rotation angle sent by terminal equipment, wherein the initial vector and the gesture rotation angle are used for representing gesture information of the terminal equipment;
Performing conversion processing according to the initial vector and the attitude rotation angle to obtain a final vector of the wearable display equipment;
and calibrating a display interface of the wearable display device so as to determine a corresponding control position point on the display interface according to the final vector.
Preferably, the acquiring the initial vector and the attitude rotation angle sent by the terminal device includes:
acquiring initial posture information and current posture information of the terminal equipment;
selecting a corresponding initial vector on a display interface of the terminal equipment according to the initial gesture information, wherein the initial vector has a starting point which is an origin of a coordinate system corresponding to the display interface of the terminal equipment and is perpendicular to the display interface of the terminal equipment;
and determining the gesture rotation angle corresponding to the gesture change according to the initial gesture information and the current gesture information, wherein the gesture rotation angle comprises a left-right rotation angle and/or an up-down rotation angle around a spatial coordinate axis of the terminal device.
Preferably, the converting according to the initial vector and the gesture rotation angle to obtain a final vector of the wearable display device includes:
Acquiring a length value of the initial vector, the left-right rotation angle of the gesture rotation angle and/or an angle value and a direction corresponding to the up-down rotation angle;
when the angle value is any angle value of the left and right rotation angles, performing tangent function processing according to the length value of the initial vector and the angle value to obtain a first intersection point of the final vector on a virtual plane of the wearable display device;
when the angle value is any angle value of the vertical rotation angle, performing tangent function processing according to the length value of the initial vector and the angle value to obtain a second intersection point of the final vector on a virtual plane of the wearable display device;
respectively determining X, Y axis directions of the final vectors according to the directions corresponding to the angle values;
and determining the final vector according to the X, Y axis direction of the final vector, the corresponding first intersection point and second intersection point and the end point of the initial vector.
Preferably, the converting according to the initial vector and the gesture rotation angle to obtain a final vector of the wearable display device includes:
acquiring display interface parameters of the wearable display equipment;
Determining a rotation angle limit value of the final vector according to the relation between the display interface parameter and the origin corresponding to the initial vector;
determining a unit displacement parameter corresponding to the final vector according to the relation among the display interface parameter, the position information of the origin on the display interface of the wearable display device and the rotation angle limit value, wherein the unit displacement parameter comprises unit displacement parameters corresponding to left, right, upward and downward;
acquiring the left-right rotation angle of the rotation angle and/or the angle value and the direction corresponding to the up-down rotation angle;
determining a third intersection point, a fourth intersection point and a X, Y axis direction of the final vector on a virtual plane of the wearable display device according to the corresponding angle value, the direction and the unit displacement parameter;
and determining the final vector according to the X, Y axis direction of the final vector, the corresponding third intersection point, the fourth intersection point and the end point of the initial vector.
Preferably, the determining the rotation angle limit value of the final vector according to the relation between the display interface parameter and the origin corresponding to the initial vector includes:
Acquiring edge points corresponding to an edge interface of a display interface of the wearable display device;
and determining the rotation angle limit value of the corresponding final vector according to the relation among the origin, the edge point and the length value of the initial vector.
Preferably, the determining the unit displacement parameter corresponding to the final vector according to the relationship among the display interface parameter, the position information of the origin on the display interface of the wearable display device, and the rotation angle limit value includes:
acquiring current position information corresponding to the width and/or length of the origin relative to the display interface;
determining a movement parameter according to the current position information and a numerical relation corresponding to the width and/or the length of the display interface;
and determining the corresponding unit displacement parameter according to the relation between the movement parameter and the corresponding rotation angle limit value.
Preferably, the calibrating the display interface of the wearable display device includes:
acquiring plane parameters corresponding to a virtual plane of the wearable display equipment and interface parameters corresponding to a display interface;
normalizing a display interface of the wearable display device according to the plane parameter and the interface parameter to complete calibration;
Correspondingly, the determining the location point of the corresponding control on the display interface of the wearable display device according to the final vector includes:
acquiring coordinate information of the final vector on a display interface of the wearable display device;
and taking the coordinate information as the position point of a display interface for controlling the wearable display device.
In order to solve the technical problem, the present invention further provides an interaction device of a wearable display device, including:
the system comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring an initial vector and a gesture rotation angle sent by terminal equipment, wherein the initial vector and the gesture rotation angle are used for representing gesture information of the terminal equipment;
the conversion processing module is used for carrying out conversion processing according to the initial vector and the gesture rotation angle to obtain a final vector of the wearable display device;
and the determining module is used for calibrating the display interface of the wearable display device so as to determine the position point corresponding to the control on the display interface according to the final vector.
In order to solve the above technical problem, the present invention further provides a wearable display device, including:
a memory for storing a computer program;
A processor for implementing the steps of the interaction method of the wearable display device as described above when executing the computer program.
To solve the above technical problem, the present invention further provides a computer readable storage medium, on which a computer program is stored, which when executed by a processor, implements the steps of the interaction method of the wearable display device as described above.
The invention provides an interaction method of wearable display equipment, which comprises the following steps: acquiring an initial vector and a gesture rotation angle sent by terminal equipment, wherein the initial vector and the gesture rotation angle are used for representing gesture information of the terminal equipment; performing conversion processing according to the initial vector and the attitude rotation angle to obtain a final vector of the wearable display equipment; and calibrating the display interface of the wearable display device so as to determine the position point corresponding to the control on the display interface of the wearable display device according to the final vector. The method utilizes the initial vector and the attitude rotation angle acquired by the existing terminal equipment, and converts the initial vector and the rotation angle to obtain a final vector, namely, a virtual ray in the wearable display equipment is consistent with the rotation direction and the rotation angle of the terminal equipment, and the operation of the wearable display equipment is realized through a control point existing between the virtual ray and a display interface of the wearable display equipment to replace the actual operation of the existing additional remote control device. The whole process does not need to add extra remote control equipment, only needs to be controlled through the existing terminal equipment of the user, saves cost, is convenient to carry, and improves the experience of the user.
In addition, the invention also provides an interaction device of the wearable display device, the wearable display device and a medium, and the interaction device and the medium have the same beneficial effects as the interaction method of the wearable display device.
Drawings
For a clearer description of embodiments of the present invention, the drawings that are required to be used in the embodiments will be briefly described, it being apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to the drawings without inventive effort for those skilled in the art.
Fig. 1 is a flowchart of an interaction method of a wearable display device according to an embodiment of the present invention;
fig. 2 is a schematic diagram of a coordinate system of a terminal device and a wearable display device according to an embodiment of the present invention;
fig. 3 is a schematic diagram of a virtual plane coordinate system of a wearable display device according to an embodiment of the present invention;
fig. 4 is a schematic diagram of a display interface coordinate system of a wearable display device according to an embodiment of the present invention;
fig. 5 is a schematic rotation diagram of a wearable display device according to an embodiment of the present invention;
fig. 6 is a schematic diagram of a rotation angle limit value of a wearable display device according to the present embodiment;
FIG. 7a is a schematic diagram illustrating a calibration conversion method according to an embodiment of the present invention;
FIG. 7b is a schematic diagram illustrating a calibration conversion method according to an embodiment of the present invention;
FIG. 7c is a schematic diagram illustrating a calibration conversion method according to an embodiment of the present invention;
FIG. 7d is a schematic diagram illustrating a calibration conversion method according to an embodiment of the present invention;
FIG. 8 is a flowchart of another method for interaction of a wearable display device according to an embodiment of the present invention;
fig. 9 is a structural diagram of an interaction device of a wearable display apparatus according to an embodiment of the present invention;
fig. 10 is a block diagram of a wearable display device according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by a person of ordinary skill in the art without making any inventive effort are within the scope of the present invention.
The invention provides an interaction method and device of a wearable display device, the wearable display device and a medium. The whole process does not need to add extra remote control equipment, only needs to be controlled through the existing terminal equipment of the user, saves cost, is convenient to carry, and improves the experience of the user.
In order to better understand the aspects of the present invention, the present invention will be described in further detail with reference to the accompanying drawings and detailed description.
It should be noted that, the wearable display device may create a virtual image in a single-eye or binocular field of view, and the near-eye display is a technique of reconstructing a virtual scene in front of the eyes by rendering light field information to the eyes through the display device placed in a non-apparent viewing distance of the eyes. The wearable display device may be, without limitation, augmented Reality (VR) glasses, virtual Reality (Augmented Reality, AR) glasses, mixed Reality (MR) glasses, extended Reality (XR) glasses, and the like. For the terminal device, the user can use the existing mobile phone, other wearable devices and the like, and the terminal device is provided with a sensor such as a gyroscope and the like to measure the data of a device (Inertial measurement unit, IMU) for measuring the three-axis attitude angle and the acceleration of the object, and the data are sent to the wearable display device.
Fig. 1 is a flowchart of an interaction method of a wearable display device according to an embodiment of the present invention, where, as shown in fig. 1, the method includes:
s11: acquiring an initial vector and a gesture rotation angle sent by terminal equipment, wherein the initial vector and the gesture rotation angle are used for representing gesture information of the terminal equipment;
S12: performing conversion processing according to the initial vector and the attitude rotation angle to obtain a final vector of the wearable display equipment;
s13: and calibrating the display interface of the wearable display device so as to determine the position point corresponding to the control on the display interface according to the final vector.
It can be understood that the wearable display device can acquire the initial vector and the gesture rotation angle sent by the terminal device by establishing communication connection with the terminal device, and the communication connection between the two devices can be a network or a bluetooth function of the two devices, or can be other communication connection modes, and the two devices are networked to transmit data measured by the gyroscope sensor of the terminal device to the wearable display device.
The initial vector and the gesture rotation angle sent by the terminal equipment are measured through an IMU device of the terminal equipment, the IMU comprises a triaxial gyroscope and a triaxial accelerometer, and the measured IMU data can be calculated to obtain the corresponding initial vector and gesture rotation angle through an algorithm formula. The corresponding algorithm may be a gesture resolving algorithm, for example, 3DoF, or may be other gesture resolving algorithms, which are not limited herein.
As an embodiment, the acquiring the initial vector and the attitude rotation angle sent by the terminal device in step S11 includes:
acquiring initial posture information and current posture information of terminal equipment;
selecting a corresponding initial vector on a display interface of the terminal equipment according to the initial gesture information, wherein the starting point of the initial vector is the origin of a coordinate system corresponding to the display interface of the terminal equipment and is perpendicular to the display interface of the terminal equipment;
and determining a gesture rotation angle corresponding to the gesture change according to the initial gesture information and the current gesture information, wherein the gesture rotation angle comprises a left-right rotation angle and/or an up-down rotation angle around a spatial coordinate axis of the terminal equipment.
Specifically, initial posture information and current posture information of the terminal equipment are obtained, and corresponding initial vectors are selected on a display interface of the terminal equipment according to the initial posture information. It will be appreciated that the terminal device differs from the wearable display device in that the wearable display device is equivalent to erecting the screen of the terminal device perpendicular to the earth's surface, seen in the visual direction, equivalent to rotating the entire coordinate system of the terminal device 90 ° around the X-axis out of the screen direction, seen in the visual direction. Fig. 2 is a schematic diagram of coordinate systems of a terminal device and a wearable display device according to an embodiment of the present invention, where, as shown in fig. 2, a vector pointing to the outside of a screen along a Y axis on the left side of fig. 2 is an initial vector of the terminal device, and in the process of converting on the right side of fig. 2, the initial vector is taken as an origin point of a coordinate system corresponding to a display interface of the terminal device, and a terminal of the vector and the vector are perpendicular to the display interface and on a Z axis.
The attitude rotation angle corresponding to the attitude change is determined by the initial attitude information and the current attitude information, and the angle may be determined by some algorithm with respect to the determination process. The gesture rotation angle is mainly a left-right rotation angle and/or an up-down rotation angle around a spatial coordinate axis of the terminal equipment. It should be noted that, the attitude rotation angle has a mapping relationship with the roll angle, yaw angle and pitch angle in the space coordinate system of the aircraft, and since the space coordinate systems corresponding to the terminal device and the wearable display device are different, the corresponding rotation angles are different, so in this embodiment, the attitude rotation angle is mainly a left-right rotation angle around the space coordinate axis of the terminal device and an up-down rotation angle, the left-right rotation angle is a Z axis around the space coordinate axis of the terminal device, and the up-down rotation angle is an X axis around the space coordinate axis of the terminal device. The rotation angle of each posture rotation angle may be any angle that exists in the left-right direction, the up-down direction, or both the left-right direction and the up-down direction, and is not limited to this, and may be set according to actual conditions.
And converting according to the initial vector and the attitude rotation angle to obtain a final vector of the wearable display device. It can be understood that the wearable display device rotates the terminal device, and finally converts the rotation into a virtual ray on the wearable display device along with the movement and rotation of the terminal device through the conversion of the vector and the angle, that is, the final vector is consistent with the rotation direction and the angle of the terminal device. The corresponding conversion processing is different from the intersection point method of the vector and the plane, and in the embodiment, the conversion processing is to determine the final vector according to the gesture rotation angle on the basis of the initial vector so as to determine the intersection point of the final vector and the display interface of the wearable display device.
Specifically, the conversion process may be to disassemble the attitude rotation angle into a left-right rotation angle and an up-down rotation angle, and the coordinate value of the final vector may be determined by triangle tangent function processing, or the coordinate value of the final vector may be determined by calculating each unit displacement, where the unit displacement is set at the display interface according to the origin, and the corresponding displacement parameters are different, mainly according to the relationship between the unit displacement determined by the rotation angle (left, right, up-down) of each maximum limit and the actual rotation angle, and the final vector may be determined according to the coordinate value. The present invention is not limited by what kind of conversion process is adopted, and the conversion process may be set according to actual conditions.
After the final vector is obtained, because the virtual plane of the wearable display device is different from the screen size of the system, the virtual plane and the screen size need to be calibrated, and the calibration process is normalized, fig. 3 is a schematic diagram of a virtual plane coordinate system of the wearable display device provided by the embodiment of the invention, and fig. 4 is a schematic diagram of a display interface coordinate system of the wearable display device provided by the embodiment of the invention, as shown in fig. 3 and 4, the 0 point of the space coordinate system is equal to the 0 point of the screen coordinate, and the calibration process is needed to calibrate the size so that the rotation angle of the two devices and the intersection point of the vector are consistent when moving. When the final vector on the virtual plane is obtained, a collision intersection point exists between the final vector and the display interface of the wearable display device, and the intersection point is used as a position point of response control of the wearable display device.
The spatial coordinates in fig. 3 are coordinate systems in which plane simulation of the wearable display device is imagined as one coordinate system, and the X, Y axis ranges in the coordinate system are [ -1,1], and fig. 4 is coordinate parameters of an operating system of the wearable display device itself, for example, pixel coordinates of an android system, and the upper left corner corresponds to (0, 0), which form a corresponding relationship. For a clearer understanding, two images can be combined into one image. For example, the resolution of the wearable device is 3840×1080, according to the coordinates of fig. 4, the upper left corner (0, 0) corresponds to (-1, 1) of the left graph, the lower right corner (3840, 1080) corresponds to (1, -1) of the left graph, the center point (1920, 540) corresponds to (0, 0) of fig. 3, and similarly, the resolution of the wearable device is 1920×1080, the correspondence is that the center point (960, 640) corresponds to (0, 0), (1920, 1080) corresponds to (1, -1). It should be noted that, regarding to the difference of the corresponding operating systems of the wearable display devices, the corresponding screen sizes are also different.
The interaction method of the wearable display equipment provided by the embodiment of the invention comprises the following steps: acquiring an initial vector and a gesture rotation angle sent by terminal equipment, wherein the initial vector and the gesture rotation angle are used for representing gesture information of the terminal equipment; performing conversion processing according to the initial vector and the attitude rotation angle to obtain a final vector of the wearable display equipment; and calibrating the display interface corresponding to the wearable display equipment so as to determine the position point corresponding to the control on the display interface of the wearable display equipment according to the final vector. The method utilizes the initial vector and the attitude rotation angle acquired by the existing terminal equipment, and converts the initial vector and the rotation angle to obtain a final vector, namely, a virtual ray in the wearable display equipment is consistent with the rotation direction and the rotation angle of the terminal equipment, and the operation of the wearable display equipment is realized through a control point existing between the virtual ray and a display interface of the wearable display equipment to replace the actual operation of the existing additional remote control device. The whole process does not need to add extra remote control equipment, only needs to be controlled through the existing terminal equipment of the user, saves cost, is convenient to carry, and improves the experience of the user.
On the basis of the above embodiment, the conversion processing in step S12 according to the initial vector and the rotation angle obtains the final vector of the wearable display device, including:
acquiring a length value of an initial vector, a left-right rotation angle of a gesture rotation angle and/or an angle value and a direction corresponding to an up-down rotation angle;
when the angle value is any angle value of the left and right rotation angles, performing tangent function processing according to the length value of the initial vector and the angle value to obtain a first intersection point of the final vector on a virtual plane of the wearable display device;
when the angle value is any angle value of the vertical rotation angle, performing tangent function processing according to the length value of the initial vector and the angle value to obtain a first intersection point of the final vector on the virtual plane of the wearable display device;
respectively determining X, Y axis directions of the final vectors according to directions corresponding to the angle values;
the final vector is determined from the X, Y axis direction of the final vector, the corresponding first and second intersection points, and the end point of the initial vector.
Specifically, since the length value of the current initial vector is fixed, when the angle value is any one of the left and right rotation angles of the gesture rotation angle, the angle value may be mainly the left rotation angle or the right rotation angle, and the tangent function processing of the triangle is performed according to the length value and the angle value of the initial vector to obtain the first intersection point of the final vector on the virtual plane of the wearable display device, so that the X value of the final vector on the plane can be known. The end point of the initial vector is used as the starting point of the final vector, and the corresponding angle value is rotated by the starting point, so that the specific value at the XOY interface can be obtained. In this embodiment, the posture rotation angle value is disassembled into a left-right rotation angle and/or an up-down rotation angle. That is, if the posture rotation angle is rotated 5 ° to the left, the posture rotation angle is changed along the coordinate axis, which corresponds to the present embodiment, and the first intersection point of the virtual plane of the wearable display device is obtained through tangent function processing, so that the X value of the final vector in the plane can be known.
If the angle is rotated downwards by 5 degrees, the second intersection point of the virtual plane of the wearable display device is obtained in the same way as the tangent function processing, and the Y value of the final vector in the plane can be obtained. If the rotation is right and upward at the same time, the corresponding X, Y value can be obtained through twice tangent function processing.
Fig. 5 is a schematic rotation diagram of a wearable display device according to an embodiment of the present invention, as shown in fig. 5, in an initial vector
Rotate the corresponding angle value to the final vector +.>
When the final vector is ++>
The rotation direction of (a) is divided into two steps, the first step is a left-right rotation angle a, and the second step is an up-down rotation angle b. Assuming that the angle a of rotation to the right of the screen is positive, the left angle a is negative, the upward angle b is positive, and the downward angle b is negative, the final vector +.>
Coordinate value of endpoint P of (2)
The X, Y axis direction of the final vector, i.e., which quadrant of the display interface the P-point is located in, can be determined with respect to the direction to which the angle value corresponds.
As another embodiment, the converting in step S12 according to the initial vector and the attitude rotation angle to obtain the final vector of the wearable display apparatus includes:
acquiring display interface parameters of wearable display equipment;
Determining a rotation angle limit value of a final vector according to the relation between the display interface parameter and the origin corresponding to the initial vector;
determining a unit displacement parameter corresponding to the final vector according to the relation among the display interface parameter, the position information of the origin on the display interface of the wearable display device and the rotation angle limit value, wherein the unit displacement parameter comprises unit displacement parameters corresponding to the left, right, upward and downward;
acquiring a left-right rotation angle of the gesture rotation angle and/or an angle value and a direction corresponding to the up-down rotation angle;
determining a third intersection point, a fourth intersection point and a X, Y axial direction of a final vector on a virtual plane of the wearable display device corresponding to the display interface according to the corresponding angle value, direction and unit displacement parameter;
the final vector is determined according to the X, Y axis direction of the final vector, the corresponding third and fourth intersection points and the end point of the initial vector.
Specifically, the corresponding relation between the posture rotation angle of the terminal device and the intersection point displacement of the final vector is a unit displacement which changes by 1 degree per rotation angle, and the third intersection point and the fourth intersection point of the final vector and the virtual plane of the wearable display device can be determined according to the unit displacement and the actual rotation angle, namely the X, Y value of the final vector in the plane is obtained.
As one embodiment, determining the rotation angle limit of the final vector from the relationship of the display interface parameter to the origin corresponding to the initial vector includes:
acquiring edge points corresponding to an edge interface of a display interface of the wearable display device;
and determining the rotation angle limit value of the corresponding final vector according to the relation among the origin, the edge point and the length value of the initial vector.
Correspondingly, since the initial vector is perpendicular to the display interface and the starting point is located at the origin, the angle corresponding to the line formed from the end point of the initial vector to the edge point of the display interface is the rotation angle limit value, and since the rotation angle limit value is the edge point corresponding to the display interface, the corresponding limit value angles are different and the same. In this embodiment, the rotation angle maximum value is referred to.
The display interface parameters can be the length and the width of the display interface, and the unit displacement parameters are determined according to the relationship among the display interface parameters, the position information of the origin on the display interface of the wearable display device and the rotation angle limit value. It can be understood that the unit displacement parameter varies according to the position of the origin and the corresponding horizontal and vertical directions, and the corresponding specific values thereof also vary.
As an embodiment, determining a unit displacement parameter corresponding to the final vector according to a relationship among a display interface parameter, position information of an origin on a display interface of the wearable display device, and a rotation angle limit value includes:
acquiring current position information corresponding to the affiliated width and/or affiliated length of the origin relative to the display interface;
determining a movement parameter according to the current position information and a numerical relation corresponding to the width and/or the length of the display interface;
and determining the corresponding unit displacement parameter according to the relation between the movement parameter and the corresponding rotation angle limit value.
Specifically, the unit displacement causes a difference in displacement due to different directions of the attitude rotation angle and the position distribution of the origin on the display interface. If the display interface is square and the origin is at the right center of the display interface, fig. 6 is a schematic diagram of the rotation angle limit of the wearable display device according to the embodiment, and as shown in fig. 6, the corresponding unit displacement parameters in the left-right up-down direction are the same. So the unit displacement dx=width/2/a, dy=height/2/a. A is the rotation angle limit value. If the display interface is rectangular, and the origin is at the right center of the display interface, the corresponding unit displacement parameters in the left-right direction are the same, and the unit displacement parameters in the up-down direction are the same. If the origin is not at the right center of the display interface, the width of the display interface is 3, the length is 4, the origin is 1/3 of the width of the display interface, the unit displacement in the left turning direction is dx=width/3/A1, the unit displacement in the right turning direction is dx=width/3*2/A2, the unit displacement in the down turning direction is dy=height/4*3/A3, and the unit displacement in the up turning direction is dy=height/4/A4.
The X, Y value of the final vector can be determined according to the unit displacement parameter, the angle value and the direction, for example, rotated 5 ° to the left, and in combination with the above example (the width of the display interface is 3, the length is 4, the origin is 1/3 of the width of the display interface, and the length is 1/4), the corresponding X value is dx×5° =width/3/a1×5 °.
From the determined X value, Y value, and X, Y axis direction, it may be determined what quadrant of the display interface is to determine the coordinates of the final vector at the display interface.
According to the embodiment, the final vector is obtained through two conversion processing modes and is consistent with the rotation direction and angle of the terminal equipment, so that the position points are obtained later conveniently, and the rotation operation of the terminal equipment is mapped to the wearable display equipment.
On the basis of the above embodiment, the calibrating the display interface of the wearable display device in step S13 includes:
obtaining plane parameters corresponding to a virtual plane of the wearable display equipment and interface parameters corresponding to a display interface;
normalizing the display interface of the wearable display equipment according to the plane parameters and the interface parameters to complete calibration;
correspondingly, determining the position point of the corresponding control on the display interface of the wearable display device according to the final vector comprises:
Acquiring coordinate information of a final vector on a display interface of wearable display equipment;
and taking the coordinate information as a position point of a display interface for controlling the wearable display device.
Specifically, since the virtual plane of the wearable display device is different from the display interface, the coordinates of the final vector at the virtual plane of the wearable display device are mapped to the coordinate system of the display interface of the terminal device. The coordinate system in fig. 3 is only a screen size of 2X2 of the wearable display device, the coordinate system corresponding to fig. 4 is ox= (1+px) = (1-Py) = Height/2, fig. 7a is a schematic diagram of a calibration conversion method step provided by the embodiment of the present invention, fig. 7b is a schematic diagram of a calibration conversion method step provided by the embodiment of the present invention, fig. 7c is a schematic diagram of a calibration conversion method step provided by the embodiment of the present invention, fig. 7d is a schematic diagram of a calibration conversion method step provided by the embodiment of the present invention, and fig. 7 a-7 d can see that the whole calibration process is determined according to the screen size, that is, the interface parameters of the display interface.
The plane parameters of the virtual plane corresponding to the display interface of the wearable display device and the interface parameters of the display interface are obtained, and the plane parameters of the virtual plane, namely the parameters in fig. 3, are obtained, wherein the interface parameters of the display interface of the wearable display device are that the origin of coordinates in fig. 4 and fig. 7 a-7 d is located at the upper left corner, and only comprise positive width and positive length. If the screen size of the wearable display device is 4X4, the corresponding coordinate system is ox= (4/2+px) ×width/4, oy= (4/2-Py) ×height/4. The formula of the corresponding calibration process is as follows:
Ox＝(M/2+Px)*Width/M，Oy＝(N/2-Py)*Height/N
Wherein M is the abscissa length corresponding to the screen Width of the wearable display device, width is the screen Width of the wearable display device, N is the ordinate length corresponding to the screen length of the wearable display device, and Height is the screen length of the wearable display device.
And combining the Ox and the Oy into corresponding position points.
The display interface of the wearable display device is calibrated, so that the corresponding control position point is determined on the display interface of the wearable display device according to the final vector, the control point existing with the display interface of the wearable display device is realized through the virtual ray, and the wearable display device is actually operated.
As an embodiment, fig. 8 is a flowchart of another interaction method of a wearable display device provided by the embodiment of the present invention, as shown in fig. 8, a terminal device obtains IMU data through its own IMU device, converts the IMU data into an initial vector and a gesture rotation angle correspondingly, sends the initial vector and the gesture rotation angle to the wearable display device, the wearable display device receives the initial vector and the gesture rotation angle, and re-enters the direction of calculating the rotation direction according to the initial vector and the gesture rotation angle to be consistent with the direction of the wearable display device, and calculates the intersection point position of the vector corresponding to the direction and the screen coordinate.
For the description of the interaction method of the wearable display device provided by the invention, please refer to the above method embodiment, the description of the method is omitted herein, and the method has the same beneficial effects as the interaction method of the wearable display device.
The invention further discloses an interaction device of the wearable display device corresponding to the method, and fig. 9 is a structural diagram of the interaction device of the wearable display device provided by the embodiment of the invention. As shown in fig. 9, the interaction means of the wearable display apparatus includes:
the system comprises an acquisition module, a display module and a display module, wherein the acquisition module is used for acquiring an initial vector and a gesture rotation angle sent by terminal equipment, wherein the initial vector and the gesture rotation angle are used for representing gesture information of the terminal equipment;
the conversion processing module is used for carrying out conversion processing according to the initial vector and the gesture rotation angle to obtain a final vector of the wearable display device;
the determining module is used for calibrating the display interface of the wearable display device so as to determine the position point corresponding to the control on the display interface of the wearable display device according to the final vector.
Since the embodiments of the device portion correspond to the above embodiments, the embodiments of the device portion are described with reference to the embodiments of the method portion, and are not described herein.
For the introduction of the interaction device of the wearable display device provided by the invention, please refer to the above method embodiment, the description of the method is omitted herein, and the method has the same beneficial effects as the interaction method of the wearable display device.
Fig. 10 is a structural diagram of a wearable display device according to an embodiment of the present invention, as shown in fig. 10, where the device includes:
a memory 21 for storing a computer program;
a processor 22 for implementing the steps of the interaction method of the wearable display device when executing the computer program.
The wearable display device provided by the present embodiment may include, but is not limited to, VR glasses, AR glasses, MR glasses, XR glasses, and the like.
Processor 22 may include one or more processing cores, such as a 4-core processor, an 8-core processor, or the like, among others. The processor 22 may be implemented in hardware in at least one of a digital signal processor (Digital Signal Processor, DSP), a Field programmable gate array (Field-Programmable Gate Array, FPGA), a programmable logic array (Programmable Logic Array, PLA). The processor 22 may also include a main processor, which is a processor for processing data in an awake state, also referred to as a central processor (Central Processing Unit, CPU), and a coprocessor; a coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 22 may be integrated with an image processor (Graphics Processing Unit, GPU) for use in responsible for rendering and rendering of content required for display by the display screen. In some embodiments, the processor 22 may also include an artificial intelligence (Artificial Intelligence, AI) processor for processing computing operations related to machine learning.
Memory 21 may include one or more computer-readable storage media, which may be non-transitory. Memory 21 may also include high-speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In this embodiment, the memory 21 is at least used for storing a computer program 211, where the computer program, when loaded and executed by the processor 22, can implement the relevant steps of the interaction method of the wearable display device disclosed in any of the foregoing embodiments. In addition, the resources stored in the memory 21 may further include an operating system 212, data 213, and the like, and the storage manner may be transient storage or permanent storage. The operating system 212 may include Windows, unix, linux, among other things. The data 213 may include, but is not limited to, data related to the interaction method of the wearable display device, etc.
In some embodiments, the wearable display device may further include a display screen 23, an input-output interface 24, a communication interface 25, a power supply 26, and a communication bus 27.
Those skilled in the art will appreciate that the structure shown in fig. 10 is not limiting of a wearable display device and may include more or fewer components than illustrated.
The processor 22 implements the interaction method of the wearable display apparatus provided in any of the above embodiments by calling instructions stored in the memory 21.
For the introduction of the wearable display device provided by the invention, please refer to the above method embodiment, the invention is not described herein, and the method has the same beneficial effects as the interaction method of the wearable display device.
Further, the present invention also provides a computer readable storage medium having a computer program stored thereon, which when executed by the processor 22 implements the steps of the interaction method of the wearable display device as described above.
It will be appreciated that the methods of the above embodiments, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored on a computer readable storage medium. Based on this understanding, the technical solution of the present invention may be embodied essentially or in part or all of the technical solution or in part in the form of a software product stored in a storage medium for performing all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
For the introduction of the computer readable storage medium provided by the present invention, please refer to the above method embodiment, the present invention is not described herein, and the method has the same advantages as the interaction method of the wearable display device.
The interaction method of the wearable display equipment, the interaction device of the wearable display equipment and the medium provided by the invention are described in detail. In the description, each embodiment is described in a progressive manner, and each embodiment is mainly described by the differences from other embodiments, so that the same similar parts among the embodiments are mutually referred. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant points refer to the description of the method section. It should be noted that it will be apparent to those skilled in the art that various modifications and adaptations of the invention can be made without departing from the principles of the invention and these modifications and adaptations are intended to be within the scope of the invention as defined in the following claims.
It should also be noted that in this specification, relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
Claims (10)
1. A method of interaction for a wearable display device, comprising:
acquiring an initial vector and a gesture rotation angle sent by terminal equipment, wherein the initial vector and the gesture rotation angle are used for representing gesture information of the terminal equipment;
performing conversion processing according to the initial vector and the attitude rotation angle to obtain a final vector of the wearable display equipment;
and calibrating a display interface of the wearable display device so as to determine a corresponding control position point on the display interface according to the final vector.
2. The interaction method of the wearable display apparatus according to claim 1, wherein the obtaining the initial vector and the gesture rotation angle sent by the terminal apparatus includes:
acquiring initial posture information and current posture information of the terminal equipment;
selecting a corresponding initial vector on a display interface of the terminal equipment according to the initial gesture information, wherein the initial vector has a starting point which is an origin of a coordinate system corresponding to the display interface of the terminal equipment and is perpendicular to the display interface of the terminal equipment;
and determining the gesture rotation angle corresponding to the gesture change according to the initial gesture information and the current gesture information, wherein the gesture rotation angle comprises a left-right rotation angle and/or an up-down rotation angle around a spatial coordinate axis of the terminal device.
3. The interaction method of the wearable display apparatus according to claim 2, wherein the converting according to the initial vector and the gesture rotation angle to obtain a final vector of the wearable display apparatus includes:
acquiring a length value of the initial vector, the left-right rotation angle of the gesture rotation angle and/or an angle value and a direction corresponding to the up-down rotation angle;
when the angle value is any angle value of the left and right rotation angles, performing tangent function processing according to the length value of the initial vector and the angle value to obtain a first intersection point of the final vector on a virtual plane of the wearable display device;
when the angle value is any angle value of the vertical rotation angle, performing tangent function processing according to the length value of the initial vector and the angle value to obtain a second intersection point of the final vector on a virtual plane of the wearable display device;
respectively determining X, Y axis directions of the final vectors according to the directions corresponding to the angle values;
and determining the final vector according to the X, Y axis direction of the final vector, the corresponding first intersection point and second intersection point and the end point of the initial vector.
4. The interaction method of the wearable display apparatus according to claim 2, wherein the converting according to the initial vector and the gesture rotation angle to obtain a final vector of the wearable display apparatus includes:
acquiring display interface parameters of the wearable display equipment;
determining a rotation angle limit value of the final vector according to the relation between the display interface parameter and the origin corresponding to the initial vector;
determining a unit displacement parameter corresponding to the final vector according to the relation among the display interface parameter, the position information of the origin on the display interface of the wearable display device and the rotation angle limit value, wherein the unit displacement parameter comprises unit displacement parameters corresponding to left, right, upward and downward;
acquiring the left-right rotation angle of the rotation angle and/or the angle value and the direction corresponding to the up-down rotation angle;
determining a third intersection point, a fourth intersection point and a X, Y axis direction of the final vector on a virtual plane of the wearable display device according to the corresponding angle value, the direction and the unit displacement parameter;
and determining the final vector according to the X, Y axis direction of the final vector, the corresponding third intersection point, the fourth intersection point and the end point of the initial vector.
5. The method of interaction of a wearable display device of claim 4, wherein the determining the rotation angle limit of the final vector from the relationship of the display interface parameter to the origin corresponding to the initial vector comprises:
acquiring edge points corresponding to an edge interface of a display interface of the wearable display device;
and determining the rotation angle limit value of the corresponding final vector according to the relation among the origin, the edge point and the length value of the initial vector.
6. The method according to claim 5, wherein determining the unit displacement parameter corresponding to the final vector according to the relationship among the display interface parameter, the position information of the origin on the display interface of the wearable display device, and the rotation angle limit value includes:
acquiring current position information corresponding to the width and/or length of the origin relative to the display interface;
determining a movement parameter according to the current position information and a numerical relation corresponding to the width and/or the length of the display interface;
and determining the corresponding unit displacement parameter according to the relation between the movement parameter and the corresponding rotation angle limit value.
7. The interaction method of the wearable display apparatus according to any one of claims 1 to 6, wherein the calibrating the display interface of the wearable display apparatus includes:
acquiring plane parameters corresponding to a virtual plane of the wearable display equipment and interface parameters corresponding to a display interface;
normalizing a display interface of the wearable display device according to the plane parameter and the interface parameter to complete calibration;
correspondingly, the determining the location point of the corresponding control on the display interface of the wearable display device according to the final vector includes:
acquiring coordinate information of the final vector on a display interface of the wearable display device;
and taking the coordinate information as the position point of a display interface for controlling the wearable display device.
8. An interactive apparatus of a wearable display device, comprising:
the system comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring an initial vector and a gesture rotation angle sent by terminal equipment, wherein the initial vector and the gesture rotation angle are used for representing gesture information of the terminal equipment;
the conversion processing module is used for carrying out conversion processing according to the initial vector and the gesture rotation angle to obtain a final vector of the wearable display device;
And the determining module is used for calibrating the display interface of the wearable display device so as to determine the position point corresponding to the control on the display interface according to the final vector.
9. A wearable display device, comprising:
a memory for storing a computer program;
a processor for implementing the steps of the interaction method of the wearable display apparatus as claimed in any of claims 1 to 7 when executing the computer program.
10. A computer readable storage medium, characterized in that it has stored thereon a computer program which, when executed by a processor, implements the steps of the interaction method of a wearable display device according to any of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310166100.0A CN116149491A (en) | 2023-02-24 | 2023-02-24 | Interaction method, device, equipment and medium of wearable display equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310166100.0A CN116149491A (en) | 2023-02-24 | 2023-02-24 | Interaction method, device, equipment and medium of wearable display equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116149491A true CN116149491A (en) | 2023-05-23 |
Family
ID=86350508
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310166100.0A Pending CN116149491A (en) | 2023-02-24 | 2023-02-24 | Interaction method, device, equipment and medium of wearable display equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116149491A (en) |
-
2023
- 2023-02-24 CN CN202310166100.0A patent/CN116149491A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107564089B (en) | Three-dimensional image processing method, device, storage medium and computer equipment | |
| US10438401B2 (en) | Caching in map systems for displaying panoramic images | |
| WO2017135129A1 (en) | Virtual space image providing method and program therefor | |
| US20170186219A1 (en) | Method for 360-degree panoramic display, display module and mobile terminal | |
| WO2015193708A1 (en) | 3d scanning with depth cameras using mesh sculpting | |
| JP5656514B2 (en) | Information processing apparatus and method | |
| US20180224945A1 (en) | Updating a Virtual Environment | |
| US11294535B2 (en) | Virtual reality VR interface generation method and apparatus | |
| JP7274510B2 (en) | VIRTUAL OBJECT DISPLAY DEVICE AND VIRTUAL OBJECT DISPLAY METHOD | |
| WO2009128181A1 (en) | Image display device, method for controlling image display device, and information storage medium | |
| CN106990836B (en) | Method for measuring spatial position and attitude of head-mounted human input device | |
| WO2020149270A1 (en) | Method for generating 3d object arranged in augmented reality space | |
| US20210183158A1 (en) | Placement and manipulation of objects in augmented reality environment | |
| US11579711B2 (en) | Three-dimensional object position tracking system | |
| CN106569696B (en) | Method and system for rendering output panoramic image and portable terminal | |
| CN110999307A (en) | Display apparatus, server, and control method thereof | |
| CN108553895A (en) | User interface element and the associated method and apparatus of three-dimensional space model | |
| CN108153417B (en) | Picture compensation method and head-mounted display device adopting same | |
| CN111078029A (en) | Air mouse information determination method, device, equipment and storage medium | |
| CN116149491A (en) | Interaction method, device, equipment and medium of wearable display equipment | |
| CN103000161A (en) | Image displaying method and device and intelligent handheld terminal | |
| JP7287172B2 (en) | Display control device, display control method, and program | |
| JP2009259130A (en) | Image display device, method for controlling the image display device, and program | |
| JP5174522B2 (en) | Image display device, control method and program for image display device | |
| CN106990838B (en) | Method and system for locking display content in virtual reality mode |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |