CN113986108A - Head-mounted display device, control method thereof, and computer-readable storage medium - Google Patents
Head-mounted display device, control method thereof, and computer-readable storage medium Download PDFInfo
- Publication number
- CN113986108A CN113986108A CN202111266595.1A CN202111266595A CN113986108A CN 113986108 A CN113986108 A CN 113986108A CN 202111266595 A CN202111266595 A CN 202111266595A CN 113986108 A CN113986108 A CN 113986108A
- Authority
- CN
- China
- Prior art keywords
- head
- mounted display
- pressure
- display device
- determining
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 57
- 230000008569 process Effects 0.000 claims description 17
- 230000006870 function Effects 0.000 claims description 11
- 238000001514 detection method Methods 0.000 description 16
- 230000003190 augmentative effect Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 239000012790 adhesive layer Substances 0.000 description 1
- 238000011982 device technology Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
- G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/017—Head mounted
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/013—Eye tracking input arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
- G06F3/0482—Interaction with lists of selectable items, e.g. menus
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G06F3/04847—Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Optics & Photonics (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The invention discloses a control method of a head-mounted display device, wherein the head-mounted display device comprises a main body and two supporting parts which extend outwards from the main body to two ends, and the main body at least comprises a lens assembly; at least one of the supporting parts is provided with a pressure sensor; the method comprises the following steps: acquiring pressure information of a preset area on the head-mounted display device through the pressure sensor; determining device control parameters of the head-mounted display device according to the pressure information; and controlling the head-mounted display equipment to operate according to the equipment control parameters. The invention also discloses a head-mounted display device and a computer readable storage medium. The invention aims to improve the accuracy of the operation and control of the display interface of the head-mounted display equipment.
Description
Technical Field
The present invention relates to the field of head-mounted display device technologies, and in particular, to a method for controlling a head-mounted display device, and a computer-readable storage medium.
Background
With the development of economic technology, the living standard of people is continuously improved, and the application of head-mounted display equipment (such as augmented reality equipment and the like) is more and more extensive. At present, a head-mounted display device is generally controlled by recognizing a user gesture, however, the gesture recognition is easily affected by factors such as scene light, user skin color and the like, so that a gesture recognition result is inaccurate, and the device cannot be accurately controlled.
Disclosure of Invention
The invention mainly aims to provide a control method of a head-mounted display device, the head-mounted display device and a computer readable storage medium, aiming at improving the control accuracy of the head-mounted display device.
In order to achieve the above object, the present invention provides a method for controlling a head-mounted display device, where the head-mounted display device includes a main body and two supporting portions extending outward from the main body, and the main body at least includes a lens assembly; at least one of the supporting parts is provided with a pressure sensor; the control method of the head-mounted display device comprises the following steps:
acquiring pressure information of a preset area on the head-mounted display device through the pressure sensor;
determining device control parameters of the head-mounted display device according to the pressure information;
and controlling the head-mounted display equipment to operate according to the equipment control parameters.
Optionally, the step of determining a device control parameter of the head-mounted display device according to the pressure information includes:
acquiring a first target identification result corresponding to the head-mounted display equipment, wherein the first target identification result comprises information whether a user has a fixation point;
and determining the equipment control parameters according to the first target recognition result and the pressure information.
Optionally, the pressure sensor comprises a first pressure sensor and a second pressure sensor, and is arranged on the same support part;
the pressure information includes first sub-pressure information detected by the first pressure sensor and second sub-pressure information detected by the second pressure sensor, and the step of determining the device control parameter according to the first target recognition result and the pressure information includes:
determining operation characteristic information of sliding operation in the preset area according to the first sub-pressure information and the second sub-pressure information;
and determining the equipment control parameters according to the operation characteristic information and the first target identification result.
Optionally, the operation feature information includes a sliding direction, and the step of determining the device control parameter according to the operation feature information and the first target recognition result includes:
when the first target recognition result indicates that the user has a fixation point, if the sliding direction is a first direction, determining that the equipment control parameter is to enlarge a display area corresponding to the fixation point; and if the sliding direction is a second direction, determining that the equipment control parameter is to reduce the display area corresponding to the watching point.
Optionally, the operation feature information includes a sliding direction, and the step of determining the device control parameter according to the operation feature information and the first target recognition result includes:
when the first target recognition result indicates that the user does not have a fixation point, if the sliding operation direction is a third direction, determining that the equipment control parameter is switched to a first display interface; and if the sliding operation direction is a fourth direction, determining that the equipment control parameter is switched to a second display interface.
Optionally, the pressure sensors include a third pressure sensor and a fourth pressure sensor, and are disposed on the same support portion; the pressure information comprises third sub-pressure information detected by the third pressure sensor and fourth sub-pressure information detected by a fourth pressure sensor, and the step of determining the device control parameter of the head-mounted display device according to the pressure information comprises the following steps:
respectively identifying preset pressure characteristic signals in the third sub-pressure information and the fourth sub-pressure information in the process of displaying an application interface of a preset application by the head-mounted display device to obtain a second target identification result; the preset pressure characteristic signal is a pressure signal which is greater than a set pressure threshold value;
determining operation characteristic information corresponding to the preset area according to the second target identification result;
and determining the equipment control parameters according to the operation characteristic information.
Optionally, the step of determining the device control parameter according to the operation feature information includes:
when the operation characteristic information comprises clicking the preset area, determining the device control parameter as a target menu for selecting the application interface;
when the operation characteristic information comprises double clicking of the preset area, determining the equipment control parameter as that the target menu is displayed on the application interface;
and when the operation characteristic information comprises that sliding operation exists in the preset area, determining the equipment control parameter as a menu currently displayed by the switching application interface.
Optionally, in the process of displaying an application interface of a preset application on the head-mounted display device, respectively identifying preset pressure characteristic signals in the third sub-pressure information and the fourth sub-pressure information, and obtaining a second target identification result further includes:
acquiring the running state of the fixation point identification function of the head-mounted display equipment; the fixation point identification function is used for identifying whether a user has a fixation point;
and when the running state is the closing state, executing the step of respectively identifying preset pressure characteristic signals in the third sub-pressure information and the fourth sub-pressure information to obtain a second target identification result in the process of displaying an application interface of a preset application by the head-mounted display device.
Further, in order to achieve the above object, the present application also proposes a head mounted display device including:
a body comprising at least a lens component;
the number of the supporting parts is two, the supporting parts extend outwards from the main body to two ends, and at least one supporting part is provided with a pressure sensor; a control device, the pressure sensor being connected with the control device, the control device comprising: the control program of the head-mounted display device comprises a memory, a processor and a control program of the head-mounted display device, wherein the control program of the head-mounted display device is stored in the memory and can run on the processor, and when the control program of the head-mounted display device is executed by the processor, the steps of the control method of the head-mounted display device are realized.
Further, in order to achieve the above object, the present application also proposes a computer-readable storage medium having stored thereon a control program of a head-mounted display device, which when executed by a processor, implements the steps of the method of controlling the head-mounted display device as recited in any one of the above.
According to the control method of the head-mounted display device, the pressure information of the preset area on the head-mounted display device is obtained to determine the device control parameters to control the operation of the head-mounted display device, and the control of the device can be completed only by pressing the preset area by a user.
Drawings
FIG. 1 is a diagram illustrating a hardware configuration involved in operation of an embodiment of a head-mounted display apparatus according to the present invention;
FIG. 2 is a schematic diagram of a pressure sensor mounting structure in an embodiment of a head-mounted display device according to the invention;
FIG. 3 is a flowchart illustrating a method for controlling a head-mounted display device according to an embodiment of the present invention;
FIG. 4 is a flowchart illustrating a method for controlling a head-mounted display device according to another embodiment of the present invention;
fig. 5 is a flowchart illustrating a method for controlling a head-mounted display device according to another embodiment of the present invention.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The main solution of the embodiment of the invention is as follows: based on a head-mounted display device, the head-mounted display device comprises a main body and two supporting parts extending outwards from the main body to two ends, wherein the main body at least comprises a lens assembly; at least one of the supporting parts is provided with a pressure sensor; acquiring pressure information of a preset area on the head-mounted display device through the pressure sensor; determining device control parameters of the head-mounted display device according to the pressure information; and controlling the head-mounted display equipment to operate according to the equipment control parameters.
In the prior art, the head-mounted display device is generally controlled by recognizing the gesture of the user, however, the gesture recognition is easily affected by factors such as scene light, user skin color and the like, so that the gesture recognition result is inaccurate, and the device cannot be accurately controlled.
The present invention provides the above solution, and aims to improve the accuracy of the operation of the head-mounted display device.
The embodiment of the invention provides a head-mounted display device. In this embodiment, the head-mounted display device is an AR device (Augmented Reality device). In other embodiments, the head-mounted display device may also be other types of head-mounted display-enabled devices, such as VR devices (Virtual Reality devices) and the like.
In this embodiment, the head-mounted display device includes a main body, two supporting portions extending outward from the main body to two ends, and a control device, where the main body at least includes a lens assembly; and a pressure sensor is arranged on at least one of the supporting parts. In particular, the support comprises a housing provided with at least one pressure sensor 1. At least one pressure sensor 1 is connected to a control device, which can acquire the data detected by the pressure sensor 1. In addition, the head-mounted display device may further include a display module 3 for displaying an interface, and the display module 3 may be connected to a control device, and the control device may be configured to control the interface displayed by the display module 3.
In the present embodiment, the number of the pressure sensors 1 provided on the same support portion is two. In other embodiments, the number of pressure sensors 1 provided on the same support portion may also be provided with a greater or lesser number, for example 1, 3, 4 or 5, etc., according to actual requirements.
Specifically, in the present embodiment, referring to fig. 1, the housing includes a first case 21 and a second case 22. The first shell 21 and the second shell 22 enclose a mounting groove therebetween, and the pressure sensor 1 is disposed in the mounting groove. Specifically, in the working process of the head-mounted display device, the first shell 21 is disposed close to the human body, the second shell 22 is disposed far away from the human body, and the pressure sensor 1 is fixed to the second shell 22. Specifically, the pressure sensor 1 may be attached to the inner wall of the second casing 22 by an adhesive layer 01 such as a double-sided tape.
Based on this, the detection data of the pressure sensor 1 can reflect the deformation condition of the shell, the area of the shell corresponding to the setting position of the pressure sensor 1 is defined as a preset area, a user executes different pressing operations in the preset area, different detection signals can be generated at the pressure sensor 1, and the different detection signals can be correspondingly associated with different equipment control parameters, so that the control device can acquire the equipment control parameters input by the user by acquiring and analyzing the detection signals of the pressure sensor 1 so as to control the display interface of the head-mounted display equipment.
Specifically, referring to fig. 2, the control apparatus of the head-mounted display device includes: a processor 1001 (e.g., a CPU), a memory 1002, a timer 1003, and the like. The memory 1002 and the timer 1003 are both connected to the processor 1001. The memory 1002 may be a high-speed RAM memory or a non-volatile memory (e.g., a disk memory). The memory 1002 may alternatively be a storage device separate from the processor 1001.
Those skilled in the art will appreciate that the configuration of the device shown in fig. 2 is not intended to be limiting of the device and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components.
As shown in fig. 2, a control program of the head-mounted display apparatus may be included in the memory 1002, which is a kind of computer-readable storage medium. In the apparatus shown in fig. 2, the processor 1001 may be configured to call a control program of the head mounted display device stored in the memory 1002, and perform operations of relevant steps of a control method of the head mounted display device in the following embodiments.
The embodiment of the invention also provides a control method of the head-mounted display equipment, which is applied to control the head-mounted display equipment.
Referring to fig. 3, an embodiment of a method for controlling a head-mounted display device according to the present application is provided. In this embodiment, the method for controlling a head-mounted display device includes:
step S10, acquiring pressure information of a preset area on the head mounted display device through the pressure sensor;
the preset area is specifically an area which is preset on the head-mounted display device and is used for detecting the pressure deformation condition of the head-mounted display device through a pressure sensor. When the preset area is deformed under pressure, a corresponding detection signal can be formed on the pressure sensor of the head-mounted display device, and the pressure information can be obtained by acquiring the detection signal of the pressure sensor.
Different pressing operations on the preset area can correspondingly form different pressure information.
Specifically, the pressure information herein may include a pressure signal detected by the pressure sensor at a certain time, may also include a pressure signal detected by the pressure sensor at different times, and may also include a detection time when the pressure signal is detected by different pressure sensors.
Step S20, determining the device control parameters of the head-mounted display device according to the pressure information;
the device control parameter is specifically a parameter for controlling the head-mounted display device. In this embodiment, the device control parameters include interface control parameters, and the interface control parameters may specifically include any parameters for controlling the display interface, such as enlarging the display area of the display interface, reducing the display area of the display interface, switching the display interface, selecting a menu, opening a menu, and/or switching a menu. In other embodiments, the device control parameters may include any device operation-related control parameters other than interface control parameters, such as switch control parameters, audio control parameters, and the like.
Different pressure information corresponds to different device control parameters. The corresponding relation between the pressure information and the equipment control parameters can be preset, can be default configuration of the system, and can also be set by a user based on self requirements. Based on the corresponding relationship, the device control parameter corresponding to the current pressure information can be determined.
Specifically, the operation characteristic information corresponding to the preset region may be determined according to the pressure information, and the device control parameter may be determined based on the determined operation characteristic information. Different operational characteristic information may be associated with different device control parameters. The operation characteristic information may include operation characteristics of a pressing operation in any preset region, such as a single-click preset region, a double-click preset region, a sliding operation in the preset region, and/or a sliding direction in the preset region.
And step S30, controlling the head-mounted display equipment to operate according to the equipment control parameters.
When the equipment control parameters comprise the display area of the display interface, the head-mounted display equipment can be controlled to amplify the current whole display interface, or the head-mounted display equipment is controlled to amplify the target area of the current display interface;
when the equipment control parameter comprises the step of reducing the display area of the display interface, the head-mounted display equipment can be controlled to reduce the current whole display interface, or the head-mounted display equipment is controlled to reduce the target area of the current display interface;
when the device control parameters comprise a display interface switching state, the head-mounted display device can be controlled to be switched to a designated interface from a current interface, or the head-mounted display device can be controlled to be switched to a target interface meeting preset conditions (such as a preset interface display sequence) from the current interface;
when the equipment control parameters comprise a selection menu, the head-mounted display equipment can be controlled to execute selection operation among a plurality of menus, or the head-mounted display equipment can be controlled to select a preset menu;
when the equipment control parameters comprise an opening menu, the head-mounted display equipment can be controlled to open a menu interface corresponding to the menu selected on the current display interface, and the display area of the menu interface can be smaller than or equal to the current display interface;
when the device control parameters include a switching menu, the head-mounted display device may be controlled to switch from the menu interface displayed on the current display interface to a display designated menu interface, or the head-mounted display device may be controlled to switch from the menu interface displayed on the current display interface to display a menu interface satisfying a preset condition (for example, a preset interface display sequence).
According to the control method of the head-mounted display device, the process of controlling the display interface of the head-mounted display device by the device control parameters is determined by obtaining the pressure information of the preset area on the head-mounted display device, and the control of the display interface can be completed only by pressing the preset area by a user. When the equipment control parameters are interface control parameters, the accuracy of interface control of the head-mounted display equipment can be effectively improved.
Further, based on the above embodiments, another embodiment of the method for controlling a head-mounted display device of the present application is provided. In this embodiment, referring to fig. 4, the step S20 includes:
step S21, acquiring a first target recognition result corresponding to the head-mounted display device, wherein the first target recognition result includes information whether the user has a fixation point;
specifically, the eye image data of the user using the head-mounted display device can be acquired in real time through an image acquisition module on the head-mounted display device, and the acquired eye image data is processed by adopting a preset gaze point identification algorithm to judge whether the user using the head-mounted display device currently has a gaze point, so that the first target identification result is obtained.
The user presence point of regard here specifically means that the magnitude of the user's motion is less than or equal to a set magnitude threshold.
Step S22, determining the device control parameter according to the first target recognition result and the pressure information.
Different first target recognition results and different pressure information may correspond to different device control parameters.
Specifically, in this embodiment, different first target recognition results are associated with different corresponding relationships between the pressure information and the device control parameter, based on which, the current corresponding relationship between the pressure information and the device control parameter can be obtained through the current first target recognition result, and the current device control parameter of the head-mounted display device can be determined through the obtained corresponding relationship and the obtained pressure information.
In addition, in another embodiment, a fixed correspondence relationship among the first target recognition result, the pressure information, and the device control parameter may be preset, and the current device control parameter of the head-mounted display device may be obtained by substituting the current first target recognition result and the pressure information into the fixed correspondence relationship.
In this embodiment, in the process of displaying the interface on the head-mounted display device, since the control requirements of the user on the display interface are different when the user has a fixation point and does not have the fixation point, based on this, the device control parameters are determined by combining the first target identification result and the pressure information, so that the interface display process of the head-mounted display device can meet the different control requirements of the user, and the accuracy of controlling the display interface of the head-mounted display device is further improved.
Further, in this embodiment, the pressure sensor includes a first pressure sensor and a second pressure sensor, and is disposed on the same support portion; the pressure information includes first sub-pressure information detected by the first pressure sensor and second sub-pressure information detected by the second pressure sensor. Specifically, the preset area has a first position and a second position, the first position may be correspondingly provided with a first pressure sensor for detecting a pressing operation of the first position, the second position may be correspondingly provided with a second pressure sensor for detecting a pressing operation of the second position. Based on this, pressure data detected by the first pressure sensor may be acquired as first sub-pressure information here, and pressure data detected by the second pressure sensor may be acquired as second sub-pressure information here.
Based on this, step S22 includes:
step S221, determining operation characteristic information of sliding operation in the preset area according to the first sub-pressure information and the second sub-pressure information;
specifically, whether the sliding operation exists in the preset area is determined according to the first sub-pressure information and the second sub-pressure information, and when the sliding operation exists in the preset area, the operation characteristic information corresponding to the sliding operation is determined according to the first sub-pressure information and the second sub-pressure information.
The operation characteristic information may specifically include a sliding time length, a sliding direction, and/or a sliding track.
Specifically, the first sub-pressure information and the second sub-pressure information may be pressure data detected by the pressure sensor in real time, the first sub-pressure information may include a first pressure value detected by the first pressure sensor and a first detection time at which the first pressure value is detected, the second sub-pressure information may include a second pressure value detected by the second pressure sensor and a second detection time at which the second pressure value is detected, based on which, and determining that preset pressure characteristic signals (pressure signals larger than a set pressure threshold value) exist in the first pressure value and the second pressure value detected at different moments in a preset time length according to the first sub-pressure information and the second sub-pressure information, wherein the preset pressure characteristic signals indicate that a user successively presses the position of the first pressure sensor and the position of the second pressure sensor, at this moment, the sliding operation in the preset area can be determined, and otherwise, the sliding operation in the preset area can be determined. When it is determined that the sliding operation exists in the preset area, a first detection time when a first pressure value of the preset pressure characteristic signal is detected is acquired as a first target time, a second detection time when a second pressure value of the preset pressure characteristic signal is detected is acquired as a second target time, if the first target time is earlier than the second target time, the sliding direction can be determined as a first direction (the first pressure sensor points to the second pressure sensor), and if the first target time is later than the second target time, the sliding direction can be determined as a second direction (the second pressure sensor points to the first pressure sensor). Further, an interval duration between the first target timing and the second target timing may be determined as the slip duration.
Step S222, determining the device control parameter according to the operation feature information and the first target identification result.
Different operation characteristic information and different first target recognition results correspond to different equipment control parameters. Specifically, different first target recognition results correspond to different corresponding relationships between the operation characteristic information and the device control parameter. That is, the first target recognition results are different, and the same operation characteristic information may correspond to different device control parameters. Based on this, the corresponding relationship between the operation characteristic information and the device control parameter is obtained through the first target recognition result, and the current device control parameter of the head-mounted display device can be determined according to the obtained corresponding relationship and the operation characteristic information.
Specifically, in this embodiment, the operation characteristic information includes a sliding direction, and based on this, the process of determining the device control parameter through the operation characteristic information and the first target recognition result is specifically as follows: when the first target recognition result indicates that the user has a fixation point, if the sliding direction is a first direction, determining that the equipment control parameter is to enlarge a display area corresponding to the fixation point; and if the sliding direction is a second direction, determining that the equipment control parameter is to reduce the display area corresponding to the watching point. When the first target recognition result indicates that the user does not have a fixation point, if the sliding operation direction is a third direction, determining that the equipment control parameter is switched to a first display interface; and if the sliding operation direction is a fourth direction, determining that the equipment control parameter is switched to a second display interface. Based on the method, when a user watches a certain position of the display interface, the watching area can be enlarged or reduced by pressing the preset area and sliding along different directions; when a user does not watch a certain position of the display interface, the current interface can be switched to different display interfaces by pressing the preset area and sliding along different directions. It should be noted that, in other embodiments, in addition to determining the device control parameter through the sliding direction in the above manner when the user has a gaze point, the device control parameter may be determined according to the sliding duration when the user does not have a gaze point, and different durations correspond to different device control parameters; or when the user has a gazing point, determining the device control parameters according to the sliding time length, wherein different time lengths correspond to different device control parameters, and when the user does not have the gazing point, determining the device control parameters according to the sliding direction.
In the embodiment, through the above manner, the user can execute different interface controls by adopting the same sliding operation in different use scenes with and without the fixation point, so that the operation accuracy of the display interface of the head-mounted display device is improved, and the user operation is simplified. Specifically, under different scenes that a user has a fixation point and does not have the fixation point, the user can realize different operations on the display interface by sliding in different directions.
Further, based on any of the above embodiments, another embodiment of the method for controlling a head-mounted display device of the present application is provided. In this embodiment, the pressure sensors include a third pressure sensor and a fourth pressure sensor, and are disposed on the same support portion; the pressure information includes third sub-pressure information detected by the third pressure sensor and fourth sub-pressure information detected by the fourth pressure sensor, where the third sub-pressure information is the same as the concept and the obtaining manner of the first sub-pressure information and the fourth sub-pressure information and the second sub-pressure information. And will not be described in detail herein.
Based on this, referring to fig. 5, the step S20 includes:
step S201, respectively identifying preset pressure characteristic signals in the third sub-pressure information and the fourth sub-pressure information in the process of displaying an application interface of a preset application by the head-mounted display device, and obtaining a second target identification result; the preset pressure characteristic signal is a pressure signal which is greater than a set pressure threshold value;
the second target recognition result may specifically include whether the third sub-pressure information and the fourth sub-pressure information have the preset pressure characteristic signal. In addition, the second target recognition result may further include a detection time when the preset pressure characteristic signal occurs in the third sub-pressure information and/or a detection time when the preset pressure characteristic signal occurs in the fourth sub-pressure information.
Step S202, determining operation characteristic information corresponding to the preset area according to the second target identification result;
the operation characteristic information may include operation characteristics of a pressing operation in any preset region, such as a single-click preset region, a double-click preset region, a sliding operation in the preset region, and/or a sliding direction in the preset region.
The different second target recognition results correspond to different operation characteristic information.
The third sub-pressure information and the fourth sub-pressure information may be pressure data detected by the pressure sensor in real time, the third sub-pressure information may include a third pressure value detected by the first pressure sensor and a third detection time at which the third pressure value is detected, and the fourth sub-pressure information may include a fourth pressure value detected by the second pressure sensor and a fourth detection time at which the fourth pressure value is detected. Based on the operation characteristic information, whether the preset pressure characteristic signal exists in the third sub-pressure information and the fourth sub-pressure information can be respectively identified, if not, no response is made, and if yes, the operation characteristic information can be further determined according to the detection time when the preset pressure characteristic signal is detected. Specifically, if a preset pressure characteristic signal exists at a third pressure value detected at a third moment in a preset time period and a fourth pressure value is detected at a fourth moment later, the fact that the user successively presses the position of the first pressure sensor and the position of the second pressure sensor indicates that the operation characteristic information is determined to be that sliding operation exists in a preset area; if a preset pressure characteristic signal exists at a third pressure value detected at the present third moment within the preset time length, a preset pressure characteristic signal exists at a third pressure value detected at the fourth moment later, or a preset pressure characteristic signal exists at a fourth pressure value detected at the present third moment within the preset time length, and a preset pressure characteristic signal exists at a fourth pressure value detected at the fourth moment later, the operation characteristic information can be determined to be a double-click preset region; if the preset pressure characteristic signal exists in the third pressure value or the fourth pressure value detected at one moment within the preset time length, the operation characteristic information can be determined to be a click preset area. In addition, the manner of determining the sliding direction based on the third sub-pressure information and the fourth sub-pressure information may be similar to the specific manner of determining the sliding direction by referring to the first sub-pressure information and the second sub-pressure information in the above embodiment, which is not described herein again.
Step S203, determining the device control parameter according to the operation characteristic information.
Different operating characteristic information corresponds to different device control parameters. Specifically, the correspondence between the operation characteristic information and the device control parameter may be obtained based on a preset application, and different preset applications may correspond to different correspondences. Or, the corresponding relationship between the operation characteristic information and the device control parameter may be obtained based on an application interface currently displayed by a preset application, and the corresponding relationship is different if the displayed application interface is different. Based on the corresponding relationship, the device control parameter corresponding to the current operation characteristic information can be determined.
In this embodiment, the device control parameters include menu control parameters of a preset application. Different operating characteristic information corresponds to different menu control parameters. Specifically, when the operation characteristic information includes clicking the preset area, determining the device control parameter as a target menu for selecting the application interface; when the operation characteristic information comprises double clicking of the preset area, determining the equipment control parameter as that the target menu is displayed on the application interface; and when the operation characteristic information comprises that sliding operation exists in the preset area, determining the equipment control parameter as a menu currently displayed by the switching application interface. And when the operation characteristic information comprises sliding operation in the preset area, the currently displayed menu can be switched to the appointed menu. In addition, when the operation characteristic information includes the sliding operation and the sliding direction in the preset area, the menu required to be switched can be further determined according to the sliding direction. Specifically, the first menu may be switched to when the sliding direction is the first direction, and the second menu may be switched to when the sliding direction is the second direction.
In other embodiments, the operation characteristic information includes that the device control parameter may be a menu interface for opening a preset application when the preset region is clicked, and the device control parameter may be a menu interface for closing a preset application when the preset region is double-clicked.
In other embodiments, the device control parameter may be other types of control parameters, such as a video playback control parameter (e.g., increase volume, decrease volume, pause playback of video, and/or start playback of video, etc.)
In this embodiment, in the above manner, in the process of displaying the application interface of the preset application on the head-mounted display device, different device control parameters related to the preset application can be identified by acquiring the pressure information of different positions in the preset area, so that the user can accurately control the application interface based on the actual application control requirement.
Further, in this embodiment, before step S201, the method further includes: acquiring the running state of the fixation point identification function of the head-mounted display equipment; the fixation point identification function is used for identifying whether a user has a fixation point; when the operation state is the off state, executing step S201; when the operation state is the on state, step S21 is executed.
Specifically, the gaze point identification function may be turned on or off by a user through an input instruction, or may be determined whether to turn on the gaze point identification function by identifying the type of the current display interface through the head-mounted display device, where the gaze point identification functions corresponding to different types are different in open/close states.
In this embodiment, the above manner can ensure that the head-mounted display device can control the display interface based on the actual requirements of the user in the specific scene with the fixation point identification function closed, so as to improve the accuracy of interface control. In addition, when the fixation point identification function is started, other schemes are adopted to determine the control parameters of the display interface, so that the head-mounted display equipment can accurately meet the control requirements of the user interface in any working scene.
Furthermore, an embodiment of the present invention further provides a computer-readable storage medium, where a control program of a head-mounted display device is stored on the computer-readable storage medium, and when executed by a processor, the control program of the head-mounted display device implements the relevant steps of any embodiment of the above method for controlling the head-mounted display device.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or system that comprises the element.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) as described above and includes instructions for enabling a terminal device (e.g., a mobile phone, a computer, a server, a head-mounted display device, or a network device) to execute the method according to the embodiments of the present invention.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (10)
1. The control method of the head-mounted display device is characterized in that the head-mounted display device comprises a main body and two supporting parts which extend outwards from the main body to two ends, wherein the main body at least comprises a lens assembly; at least one of the supporting parts is provided with a pressure sensor; the control method of the head-mounted display device comprises the following steps:
acquiring pressure information of a preset area on the head-mounted display device through the pressure sensor;
determining device control parameters of the head-mounted display device according to the pressure information;
and controlling the head-mounted display equipment to operate according to the equipment control parameters.
2. The method of claim 1, wherein the step of determining the device control parameters of the head mounted display device according to the pressure information comprises:
acquiring a first target identification result corresponding to the head-mounted display equipment, wherein the first target identification result comprises information whether a user has a fixation point;
and determining the equipment control parameters according to the first target recognition result and the pressure information.
3. The method of controlling a head-mounted display device according to claim 2, wherein the pressure sensor includes a first pressure sensor and a second pressure sensor, provided in the same support portion;
the pressure information includes first sub-pressure information detected by the first pressure sensor and second sub-pressure information detected by the second pressure sensor, and the step of determining the device control parameter according to the first target recognition result and the pressure information includes:
determining operation characteristic information of sliding operation in the preset area according to the first sub-pressure information and the second sub-pressure information;
and determining the equipment control parameters according to the operation characteristic information and the first target identification result.
4. The method of claim 3, wherein the operation characteristic information includes a sliding direction, and the step of determining the device control parameter based on the operation characteristic information and the first target recognition result comprises:
when the first target recognition result indicates that the user has a fixation point, if the sliding direction is a first direction, determining that the equipment control parameter is to enlarge a display area corresponding to the fixation point; and if the sliding direction is a second direction, determining that the equipment control parameter is to reduce the display area corresponding to the watching point.
5. The method of claim 3, wherein the operation characteristic information includes a sliding direction, and the step of determining the device control parameter based on the operation characteristic information and the first target recognition result comprises:
when the first target recognition result indicates that the user does not have a fixation point, if the sliding operation direction is a third direction, determining that the equipment control parameter is switched to a first display interface; and if the sliding operation direction is a fourth direction, determining that the equipment control parameter is switched to a second display interface.
6. The method of controlling a head-mounted display device according to any one of claims 1 to 5, wherein the pressure sensors include a third pressure sensor and a fourth pressure sensor, which are provided on the same support portion; the pressure information comprises third sub-pressure information detected by the third pressure sensor and fourth sub-pressure information detected by a fourth pressure sensor, and the step of determining the device control parameter of the head-mounted display device according to the pressure information comprises the following steps:
respectively identifying preset pressure characteristic signals in the third sub-pressure information and the fourth sub-pressure information in the process of displaying an application interface of a preset application by the head-mounted display device to obtain a second target identification result; the preset pressure characteristic signal is a pressure signal which is greater than a set pressure threshold value;
determining operation characteristic information corresponding to the preset area according to the second target identification result;
and determining the equipment control parameters according to the operation characteristic information.
7. The method of claim 6, wherein the determining the device control parameter according to the operation characteristic information comprises:
when the operation characteristic information comprises clicking the preset area, determining the device control parameter as a target menu for selecting the application interface;
when the operation characteristic information comprises double clicking of the preset area, determining the equipment control parameter as that the target menu is displayed on the application interface;
and when the operation characteristic information comprises that sliding operation exists in the preset area, determining the equipment control parameter as a menu currently displayed by the switching application interface.
8. The method for controlling a head-mounted display device according to claim 6, wherein the step of respectively identifying the preset pressure characteristic signals in the third sub-pressure information and the fourth sub-pressure information and obtaining the second target identification result during the process of displaying the application interface of the preset application by the head-mounted display device further comprises:
acquiring the running state of the fixation point identification function of the head-mounted display equipment; the fixation point identification function is used for identifying whether a user has a fixation point;
and when the running state is the closing state, executing the step of respectively identifying preset pressure characteristic signals in the third sub-pressure information and the fourth sub-pressure information to obtain a second target identification result in the process of displaying an application interface of a preset application by the head-mounted display device.
9. A head-mounted display device, comprising:
a body comprising at least a lens component;
the number of the supporting parts is two, the supporting parts extend outwards from the main body to two ends, and at least one supporting part is provided with a pressure sensor; a control device, the pressure sensor being connected with the control device, the control device comprising: memory, a processor and a control program of a head mounted display device stored on the memory and executable on the processor, the control program of the head mounted display device implementing the steps of the method of controlling a head mounted display device according to any one of claims 1 to 8 when executed by the processor.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium has stored thereon a control program of a head-mounted display device, which when executed by a processor, implements the steps of the method of controlling a head-mounted display device according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111266595.1A CN113986108A (en) | 2021-10-28 | 2021-10-28 | Head-mounted display device, control method thereof, and computer-readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111266595.1A CN113986108A (en) | 2021-10-28 | 2021-10-28 | Head-mounted display device, control method thereof, and computer-readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113986108A true CN113986108A (en) | 2022-01-28 |
Family
ID=79743869
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111266595.1A Pending CN113986108A (en) | 2021-10-28 | 2021-10-28 | Head-mounted display device, control method thereof, and computer-readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113986108A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107239222A (en) * | 2017-06-02 | 2017-10-10 | 汪念鸿 | The control method and terminal device of a kind of touch-screen |
| CN107479785A (en) * | 2017-07-31 | 2017-12-15 | 珠海市魅族科技有限公司 | A kind of control method of control, device, computer installation and readable storage medium storing program for executing |
| CN107980117A (en) * | 2016-12-22 | 2018-05-01 | 深圳市柔宇科技有限公司 | A kind of parameter adjusting method of mobile equipment, system and mobile equipment |
| US20190104798A1 (en) * | 2016-01-12 | 2019-04-11 | Ronald A. Podboy | Helmet to reduce traumatic brain injuries |
| CN109960412A (en) * | 2019-03-22 | 2019-07-02 | 北京七鑫易维信息技术有限公司 | A kind of method and terminal device based on touch-control adjustment watching area |
| CN111338546A (en) * | 2020-02-28 | 2020-06-26 | 歌尔科技有限公司 | Method for controlling head-mounted display device, terminal and storage medium |
-
2021
- 2021-10-28 CN CN202111266595.1A patent/CN113986108A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190104798A1 (en) * | 2016-01-12 | 2019-04-11 | Ronald A. Podboy | Helmet to reduce traumatic brain injuries |
| CN107980117A (en) * | 2016-12-22 | 2018-05-01 | 深圳市柔宇科技有限公司 | A kind of parameter adjusting method of mobile equipment, system and mobile equipment |
| CN107239222A (en) * | 2017-06-02 | 2017-10-10 | 汪念鸿 | The control method and terminal device of a kind of touch-screen |
| CN107479785A (en) * | 2017-07-31 | 2017-12-15 | 珠海市魅族科技有限公司 | A kind of control method of control, device, computer installation and readable storage medium storing program for executing |
| CN109960412A (en) * | 2019-03-22 | 2019-07-02 | 北京七鑫易维信息技术有限公司 | A kind of method and terminal device based on touch-control adjustment watching area |
| CN111338546A (en) * | 2020-02-28 | 2020-06-26 | 歌尔科技有限公司 | Method for controlling head-mounted display device, terminal and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10452140B2 (en) | Generation of graphical feedback in a computer system | |
| CN106406710B (en) | Screen recording method and mobile terminal | |
| US9257114B2 (en) | Electronic device, information processing apparatus,and method for controlling the same | |
| JP5387557B2 (en) | Information processing apparatus and method, and program | |
| RU2651194C2 (en) | Button interaction method and apparatus | |
| JP5709227B2 (en) | Information processing apparatus, information processing method, and program | |
| US20030095154A1 (en) | Method and apparatus for a gesture-based user interface | |
| CN108984089B (en) | Touch operation method and device, storage medium and electronic equipment | |
| CN111831205B (en) | Device control method, device, storage medium and electronic device | |
| US11604570B2 (en) | Method, device, apparatus, and storage medium for controlling touch operation mode | |
| CN113918010A (en) | Display apparatus and control method of display apparatus | |
| CN111656313A (en) | Screen display switching method, display device and movable platform | |
| JP6021488B2 (en) | Control device, control method, and control program | |
| CN113986108A (en) | Head-mounted display device, control method thereof, and computer-readable storage medium | |
| CN110908580B (en) | Method and device for controlling application | |
| CN110308821B (en) | Touch response method and electronic equipment | |
| CN111147790A (en) | Auxiliary function starting method, mobile terminal and computer readable storage medium | |
| CN116400840A (en) | Application advertisement skipping method, system, terminal and readable storage medium | |
| CN114374868A (en) | Method, apparatus, medium, and computing device for adjusting video playing speed | |
| JPH09237151A (en) | Graphical user interface | |
| CN114513690B (en) | Display device and image acquisition method | |
| CN111045509A (en) | Terminal mode switching method, terminal and vehicle | |
| CN113056724A (en) | Page skipping method and device, storage medium and electronic equipment | |
| CN110162170A (en) | Control method and device based on terminal expandable area | |
| US20240221326A1 (en) | Interactive control method, terminal device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20221122 Address after: 266104 No. 500, Songling Road, Laoshan District, Qingdao, Shandong Applicant after: GOERTEK TECHNOLOGY Co.,Ltd. Address before: 261031 workshop 1, phase III, Geer Photoelectric Industrial Park, 3999 Huixian Road, Yongchun community, Qingchi street, high tech Zone, Weifang City, Shandong Province Applicant before: GoerTek Optical Technology Co.,Ltd. |