CN117193540B

CN117193540B - Control method and system of virtual keyboard

Info

Publication number: CN117193540B
Application number: CN202311461158.4A
Authority: CN
Inventors: 罗涛; 邓訸心; 张奕静; 于入洋; 许潆心; 张雅雯; 谢莎莎; 盛玥; 郭慧
Original assignee: Southwest University of Science and Technology
Current assignee: Southwest University of Science and Technology
Priority date: 2023-11-06
Filing date: 2023-11-06
Publication date: 2024-03-12
Anticipated expiration: 2043-11-06
Also published as: CN117193540A

Abstract

The present disclosure provides a control method and a system for a virtual keyboard, which are applied to a virtual reality scene, wherein the virtual reality scene comprises a virtual keyboard, a user and a preview frame, the preview frame comprises an input text input by the user based on the virtual keyboard, the virtual keyboard comprises a delete key, and the hand of the user is positioned right above the virtual keyboard and does not touch the virtual keyboard; the method comprises the following steps: obtaining a position relation between a first position where two fingers of a user's hand are pinched and a second position of a deletion area, wherein the deletion area comprises an area where a deletion key is located and an area within a first range preset by the deletion key, and obtaining action information of the user's hand after the two fingers are pinched; and according to the position relation and the action information, performing corresponding operation on the input text so as to improve the diversity and flexibility of the control system on the virtual keyboard.

Description

Control method and system of virtual keyboard

Technical Field

The specification relates to the technical field of artificial intelligence, and can be applied to virtual reality scenes, in particular to a control method and a control system of a virtual keyboard.

Background

A virtual keyboard refers to a keyboard in a virtual reality scenario, and in the related art, when a user operates the virtual keyboard, for example, when the user performs corresponding processing on input text based on the operation on the virtual keyboard, for example, when deleting processing, the control system may delete text that has been input (i.e., input text) based on the click operation of the user on a delete key on the virtual keyboard.

However, the user's operation of the virtual keyboard may lack tactile feedback with respect to the physical keyboard in the real scene, and thus, there may be a technical problem in that accuracy is low in performing the processing of the input text based on the above method by the control system.

It should be noted that the content of the related art is only information known to the inventor, and does not represent that the information has entered the public domain before the filing date of the present disclosure, or that it may be the prior art of the present disclosure.

Disclosure of Invention

The disclosure provides a control method and a control system for a virtual keyboard, which are used for avoiding the technical problems.

In a first aspect, the present disclosure provides a control method of a virtual keyboard, which is applied to a virtual reality scene, where the virtual reality scene includes a virtual keyboard, a user, and a preview box, the preview box includes an input text input by the user based on the virtual keyboard, the virtual keyboard includes a delete key, and the hand of the user is located right above the virtual keyboard and does not touch the virtual keyboard; the method comprises the following steps:

obtaining a position relationship between a first position kneaded by two fingers of the hand of the user and a second position of a deletion area, wherein the deletion area comprises an area where the deletion key is located and an area within a first range preset by the deletion key;

Obtaining action information of the hand of the user after the two fingers are pinched; and

and carrying out corresponding operation on the input text according to the position relation and the action information.

In some embodiments, the preview pane includes a first flicker cursor therein; and performing corresponding operation on the text according to the position relation and the action information, wherein the operation comprises one of the following steps:

deleting at least part of the text to be deleted and displaying the deleted at least part of the text carrying a deletion mark under the condition that the position relation represents that the first position is located right above the second position, and the action information represents that at least part of the text in the input text is dragged after the two fingers are kneaded, and the kneaded fingers are released after the at least part of the text is dragged;

and deleting the whole word text in a preset second range of the first flicker cursor and displaying the whole word text in the preset second range carrying the deletion mark under the condition that the position relation indicates that the first position is not located right above the second position and the action information indicates that the two fingers are loosened after being kneaded twice.

In some embodiments, the method further comprises:

under the condition that at least part of the text is deleted or the whole word text in the preset second range is deleted, expanding and displaying the preview frame; and/or the number of the groups of groups,

and under the condition that the first flicker cursor moves, expanding and displaying the preview frame.

In some embodiments, the preview box includes a first blinking cursor therein, and the virtual reality scene further includes a second blinking cursor outside the preview box; the method further comprises the steps of:

in a case where the positional relationship indicates that the first position is not located directly above the second position and the motion information indicates that the two finger pinch-drags are released after being moved to a third position, controlling the first blinking cursor to move based on the two finger pinch-drags until the third position is moved to a position corresponding to the preview frame; and

the second blinking cursor is controlled to move based on the movement of the first blinking cursor.

In some embodiments, the virtual keyboard further comprises a dismissal key and/or a redo key; and

the cancel key is used for performing cancel processing on the previous operation;

the redo key is used for carrying out the redo processing on the previous operation, or the undo key and the redo key are used for carrying out the redo processing on the previous operation.

In some embodiments, the cancel key is used to cancel a previous operation, including:

in a case where a click operation of the cancel key by the user or a release after pinching of the two fingers is obtained, a cancel process is performed for the previous operation.

In some embodiments, the redo key is used to redo a previous operation, including:

and carrying out the redo processing on the previous operation in the condition that the click operation of the redo key by the user or the release after the two fingers are pinched are obtained.

In some embodiments, the dismiss key and the redo key are used to redo a previous operation, comprising:

and under the condition that the two fingers are pinched right above the withdrawal key and released after dragging a preset distance to the direction of the redo key, carrying out redo processing on the previous operation.

In some embodiments, the virtual keyboard comprises an adjusting bar thereon, wherein the adjusting bar is a transverse bar and/or a vertical bar; the adjusting bars comprise a first adjusting bar and a second adjusting bar; and

the first adjusting bar is used for adjusting the angle of the virtual keyboard;

the second adjusting bar is used for adjusting the position of the virtual keyboard;

The first adjustment bar or the second adjustment bar is used for closing the virtual keyboard.

In some embodiments, the first adjustment bar is for adjusting an angle of the virtual keyboard, comprising: controlling the virtual keyboard to change along the change of the two fingers kneading the first adjusting bar in the vertical axis direction under the condition that the two fingers kneading the first adjusting bar;

the second adjusting bar is used for adjusting the position of the virtual keyboard, and comprises the following steps: and under the condition that the second adjusting lever is selected by the two finger kneading, controlling the virtual keyboard to change along with the change of the second adjusting lever in the translational direction by the two finger kneading, wherein the translational direction comprises at least one of a horizontal axis translational direction, a vertical axis translational direction and a vertical axis translational direction.

In some embodiments, the first adjustment bar or the second adjustment bar is used to close the virtual keyboard, comprising:

controlling the virtual keyboard to be closed under the condition that the two fingers pinch the first adjusting lever or the second adjusting lever and the two pinched fingers are loosened under the preset condition;

Wherein, the preset conditions include:

the speeds of the two fingers in the vertical axis direction reach a preset first threshold value; or,

the position of the virtual keyboard in the longitudinal axis direction is smaller than a preset second threshold value.

In some embodiments, the virtual keyboard further comprises an input method switch key; the method further comprises the steps of:

under the condition that the user's finger clicks the input method switching key, switching the current input method of the virtual keyboard to a preset input method, wherein the speed of the user's finger clicking the input method switching key is changed from a speed value smaller than a preset third threshold value to a speed value larger than the preset third threshold value; and/or the number of the groups of groups,

and outputting a secondary menu comprising a plurality of input methods when the time length of the finger touch of the user to the input method switching key reaches a preset fourth threshold value or when the time length of the finger touch of the user to the input method switching key reaches a preset fifth threshold value or when the speed of the finger touch of the user to the input method switching key is smaller than a preset fifth threshold value and the acceleration of the finger touch of the user to the input method switching key changes from an acceleration value smaller than a preset eighth threshold value to an acceleration value smaller than the preset sixth threshold value and a preset cancel condition is not met within a preset time length, wherein the secondary menu is used for determining the input method clicking operation from the secondary menu to the current input method based on the secondary menu.

In a second aspect, the present disclosure provides a control system for a virtual keyboard, including:

at least one memory including at least one set of instructions to push information;

at least one processor in communication with the at least one memory;

wherein the method of any of the first aspects is implemented when the at least one processor executes the at least one set of instructions.

In a third aspect, the present disclosure provides a processor-readable storage medium storing a computer program for causing the processor to perform the method of any one of the first aspects.

In a fourth aspect, the present disclosure provides an electronic device comprising: a processor, and a memory communicatively coupled to the processor;

the memory stores computer-executable instructions;

the processor executes computer-executable instructions stored in the memory to implement the method of any one of the first aspects.

In a fifth aspect, the present disclosure provides a computer program product comprising a computer program which, when executed by a processor, implements the method of any of the first aspects.

The present disclosure provides a control method and a system for a virtual keyboard, which are applied to a virtual reality scene, wherein the virtual reality scene comprises a virtual keyboard, a user and a preview frame, the preview frame comprises an input text input by the user based on the virtual keyboard, the virtual keyboard comprises a delete key, and the hand of the user is positioned right above the virtual keyboard and does not touch the virtual keyboard; the method comprises the following steps: obtaining a position relationship between a first position of two fingers of a user's hand to pinch and a second position of a deletion area, wherein the deletion area comprises an area where a deletion key is located and an area within a first range preset by the deletion key, obtaining motion information of the user's hand after the two fingers are pinched, and performing corresponding operations on an input text according to the position relationship and the motion information, that is, a control system can perform corresponding operations such as deletion or input on the input text by combining the position relationship between the two fingers to pinch and the position of the deletion area and the motion information after the two fingers to improve the diversity and flexibility of the control system on virtual keyboard control and improve the experience of the user in a virtual reality scene.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present description, the drawings that are needed in the description of the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present description, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is an application scenario schematic diagram of a control method of a virtual keyboard according to an embodiment of the disclosure;

FIG. 2 is a schematic diagram of a control method of a virtual keyboard according to an embodiment of the disclosure;

FIG. 3 is a schematic diagram of a preview pane of an embodiment of the present disclosure;

FIG. 4 is a schematic diagram of a virtual keyboard according to one embodiment of the present disclosure;

FIG. 5 is a schematic illustration of a first blinking cursor movement in accordance with an embodiment of the present disclosure;

FIG. 6 is a schematic diagram of a virtual keyboard according to another embodiment of the present disclosure;

FIG. 7 is a schematic diagram of a virtual keyboard according to another embodiment of the disclosure;

FIG. 8 is a schematic diagram of a virtual keyboard according to another embodiment of the present disclosure;

FIG. 9 is a schematic diagram of a virtual keyboard according to another embodiment of the present disclosure;

FIG. 10 is a schematic diagram of an angle adjustment of a virtual keyboard;

FIG. 11 is a schematic diagram of a virtual keyboard according to another embodiment of the present disclosure;

FIG. 12 is a schematic diagram of a virtual keyboard according to another embodiment of the present disclosure;

FIG. 13 is a schematic diagram of a virtual keyboard according to another embodiment of the present disclosure;

fig. 14 is a hardware configuration diagram of an electronic device according to an embodiment of the present disclosure.

Detailed Description

Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary examples are not representative of all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present disclosure as detailed in the accompanying claims.

It should be understood that the terms "comprises" and "comprising," and any variations thereof, in the embodiments of the disclosure are intended to cover a non-exclusive inclusion, such that a product or apparatus that comprises a list of elements is not necessarily limited to those elements expressly listed, but may include other elements not expressly listed or inherent to such product or apparatus.

The term "and/or" in the embodiments of the present disclosure describes an association relationship of association objects, which indicates that three relationships may exist, for example, a and/or B may indicate: a exists alone, A and B exist together, and B exists alone. The character "/" generally indicates that the context-dependent object is an "or" relationship.

The term "plurality" in the embodiments of the present disclosure means two or more, and other adjectives are similar thereto.

The terms "first," "second," "third," and the like in this disclosure are used for distinguishing between similar or similar objects or entities and not necessarily for limiting a particular order or sequence, unless otherwise indicated (Unless otherwise indicated). It is to be understood that the terms so used are interchangeable under appropriate circumstances such that the embodiments of the disclosure are, for example, capable of operation in sequences other than those illustrated or otherwise described herein.

The term "unit/module" as used in this disclosure refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware or/and software code that is capable of performing the function associated with that element.

To facilitate the reader's understanding of this disclosure, at least some of the terms of this disclosure are now explained as follows:

virtual reality is a computer simulation system that creates a new interactive system through simulation of the three-dimensional world. The virtual reality utilizes a computer to generate a simulation environment, provides a user with a simulation of senses such as vision, hearing, touch and the like, allows the user to have an immersive sense and can interact in the virtual reality environment.

Virtual Reality technologies include augmented Reality (Augmented Reality, AR) technologies, virtual Reality (VR) technologies, mixed Reality (MR) technologies. Wherein,

the AR technology is a technology of skillfully fusing virtual information with a real world, and widely uses various technical means such as multimedia, three-dimensional modeling, real-time tracking and registration, intelligent interaction, sensing and the like, and applies virtual information such as characters, images, three-dimensional models, music, videos and the like generated by a computer to the real world after simulation, wherein the two kinds of information are mutually complemented, so that the real world is enhanced.

VR technology, also known as virtual reality or smart technology, includes computer, electronic information and simulation technology, and is basically realized by taking computer technology as the main component, utilizing and integrating various high-tech latest development achievements such as three-dimensional graphics technology, multimedia technology, simulation technology, display technology, servo technology and the like, and generating a realistic virtual world with various sensory experiences such as three-dimensional vision, touch sense, smell sense and the like by means of devices such as computers and the like, so that a person in the virtual world generates an immersive sense.

MR technology is an upgrade of AR technology and VR technology, and by introducing real scene information into a virtual environment, an interactive feedback information loop is built up among the virtual world, the real world and a user, so as to enhance the sense of realism of user experience. Compared with AR technology and VR technology, MR technology can better process the spatial relationship between virtual digital content and real physical world, and can enable people to interact with digital content in a more instinctive interaction mode.

The keyboard in a real scene may be referred to as a physical keyboard, or may be referred to as a physical keyboard. Accordingly, a keyboard in a virtual reality scene may be referred to as a virtual keyboard. The virtual keyboard also comprises a keyboard controlled by a user in a touch mode in the mobile terminal. The virtual keyboard aimed at by the control method of the virtual keyboard provided by the disclosure mainly refers to a non-physical keyboard in a virtual reality scene.

Taking a physical keyboard in a real scene as an example, in the related art, a control system for controlling the physical keyboard (e.g., a computer including the physical keyboard) determines input/deletion of information according to physical keys, such as determining characters (or numbers) corresponding to the physical keys as input characters (or numbers), or deleting information that has been input, in a case where a finger of a user presses a physical key (which may also be referred to as a physical key) on the physical keyboard is obtained.

In some embodiments, the control system may trigger a continuous input mode based on a long press (physical key) operation of a user's finger, e.g., the control system may determine continuously input information based on a long press operation of a physical key by a user, while the control system may trigger a "K" effect based on a long press operation of a user's finger. The "K" effect is understood to be an effect in which the information input/deletion speed is stepwise changed as the time of the long-press operation by the user's finger changes.

Taking an input operation of a virtual keyboard in a mobile terminal as an example, in the related art, the mobile terminal may output the virtual keyboard on a screen of the mobile terminal, where the virtual keyboard includes virtual keys for inputting text information, and a user may click the virtual keyboard on the screen, specifically click the virtual keys on the virtual keyboard, by using a finger, and in a case where the user's finger leaves the virtual keys, the mobile terminal displays corresponding text information on the screen or deletes the corresponding text information.

Taking an input operation of a virtual keyboard in VR (Oculus) technology as an example, in the related art, a control system may control movement of a cursor position by rays of a palm direction of a user, and determine information input of the user through the virtual keyboard according to a pinching operation of two fingers (e.g., thumb and index finger) of a user's hand.

Taking an input operation of a virtual keyboard in MR (holocens) technology as an example, in the related art, the input mode of the virtual keyboard is that a finger of a user directly contacts a virtual key of the keyboard, when the finger touches the virtual key which is not a delete key, the control system immediately inputs content corresponding to the virtual key, and when the virtual key is a delete key, the control system deletes the corresponding content.

However, in a virtual reality scenario, particularly in an MR scenario, the distance between the input box and the user is large, and it is often difficult for the user to locate a specific character and perform a corresponding operation, such as deleting the specific character.

In order to avoid at least one of the above problems, the present disclosure proposes the technical idea of the inventive effort: in a virtual reality scene including a virtual keyboard, a user and a preview box, the preview box includes an input text input by the user based on the virtual keyboard, the virtual keyboard includes a delete key, and under the condition that a hand of the user is located right above the virtual keyboard and does not touch the virtual keyboard, the control system obtains a position (for convenience of distinction, we can refer to the position as a first position) of kneading two fingers of the user and a position (for convenience of distinction, we can refer to the position as a second position) of a delete area, so as to perform corresponding processing on the input text based on the position relationship and the obtained operation (which can be referred to as action information, such as a dragging operation or an operation of loosening kneading two fingers), for example, deleting the input text.

Before explaining the implementation principle of the control method of the virtual keyboard of the present disclosure, an application scenario of the control method of the virtual keyboard of the present disclosure is described exemplarily to deepen the understanding of readers on the control method of the virtual keyboard of the present disclosure.

Fig. 1 is an application scenario schematic diagram of a control method of a virtual keyboard according to an embodiment of the present disclosure, where the control method of a virtual keyboard of the present disclosure may be applied to a control system 100 of a virtual keyboard as shown in fig. 1. As shown in fig. 1, a control system 100 of a virtual keyboard may include a target user 101, a virtual reality device 102, a server 103, and a network 104.

The target user 101 may be a user in a virtual reality scenario, and in particular may be a user using a virtual keyboard in a virtual reality scenario.

The virtual reality apparatus 102 may be a virtual reality device, an augmented reality device, or the like, and the control method of the virtual keyboard of the present disclosure may be executed on the virtual reality apparatus 102. At this time, the virtual reality device 102 may store data or instructions to perform the control method of the virtual keyboard described in this specification, and may execute or be used to execute the data or instructions. In one possible implementation solution, the virtual reality apparatus 102 may include a hardware device having a data information processing function and a program necessary for driving the hardware device to operate.

As shown in fig. 1, the virtual reality device 102 may be communicatively connected to a server 103. The server 103 may be connected to one virtual reality device 102 in communication, or may be connected to a plurality of virtual reality devices 102 in communication. In one possible implementation, the virtual reality device 102 may interact with the server 103 over the network 104 to receive or send messages, etc.

The server 103 may be a server that provides various services, for example, a background server that provides support for information processing of a plurality of accounts and sets of user data and account login information corresponding to the plurality of accounts acquired on the virtual reality device 102.

In one possible implementation solution, the control method of the virtual keyboard may be executed on the server 103. At this time, the server 103 may store data or instructions to perform the control method of the virtual keyboard described in the present specification, and may execute or be used to execute the data or instructions.

In one possible implementation solution, the server 103 may include a hardware device having a data information processing function and a program necessary for driving the hardware device to operate. Similarly, the server 103 may be communicatively connected to one virtual reality device 102 and receive data transmitted from the virtual reality device 102, or may be communicatively connected to a plurality of virtual reality devices 102 and receive data transmitted from each of the virtual reality devices 102.

The network 104 is a medium for providing a communication connection between the virtual reality device 102 and the server 103. The network 104 may facilitate the exchange of information or data. As shown in fig. 1, the virtual reality device 102 and the server 103 may be connected to a network 104, respectively, and mutually transmit information or data through the network 104.

In one possible implementation, the network 104 may be any type of wired or wireless network, or a combination thereof. For example, the network 104 may include a cable network, a wired network, a fiber optic network, a telecommunications network, an intranet, the internet, a local area network (Local Area Network, LAN), a wide area network (Wide Area Network, WAN), a wireless local area network (Wireless Local Area Network, WLAN), a metropolitan area network (Metropolitan Area Network, MAN), a public switched telephone network (Public Switched Telephone Network, PSTN), a bluetooth network (TM), a short range wireless network (zigbee (TM), a near field communication (Near Field Communication, NFC) network, or the like.

In one possible implementation, the network 104 may include one or more network access points. For example, network 104 may include a wired or wireless network access point, such as a base station or an internet switching point, through which one or more components of virtual reality device 102 and server 103 may connect to network 104 to exchange data or information.

It should be understood that the number of virtual reality devices 102, servers 103, and networks 104 in fig. 1 are merely illustrative. There may be any number of virtual reality devices 102, servers 103, and networks 104, as desired for implementation. The control method of the virtual keyboard provided by the disclosure can be completely executed on the virtual reality device 102, can be completely executed on the server 103, can be partially executed on the virtual reality device 102, and can be partially executed on the server 103.

That is, fig. 1 and the above description with respect to fig. 1 are merely for exemplarily illustrating application scenarios to which the control method of the virtual keyboard of the present disclosure may be applied, and are not to be construed as limiting the application scenarios.

Based on the technical conception, the present disclosure provides a control method of a virtual keyboard, which is applied to a virtual reality scene, wherein the virtual reality scene comprises a virtual keyboard, a user and a preview box, the preview box comprises an input text input by the user based on the virtual keyboard, the virtual keyboard comprises a delete key, and a hand of the user is positioned right above the virtual keyboard and does not touch the virtual keyboard, and fig. 2 is a schematic diagram of the control method of the virtual keyboard in the embodiment of the present disclosure. As shown in fig. 2, the method includes:

S201: a positional relationship between a first position where two fingers of a user's hand pinch, and a second position of a deletion area is obtained, wherein the deletion area includes an area where a deletion key is located, and an area within a first range preset by the deletion key.

The execution body of the embodiment of the present disclosure may be a control system (hereinafter simply referred to as a control system) of a virtual keyboard, where the control system may be a virtual reality device as described in the application scenario, may be a server as described in the application scenario, or may include a device obtained by combining the virtual reality device and the server as described in the application scenario, and the embodiment is not limited.

The two fingers may be a thumb and an index finger, for example, the control system may monitor whether the thumb and the index finger are pinched, and if so, the control system determines the position of the thumb and the index finger when pinched to obtain a first position, and then the control system determines the association between the first position and the second position in position to obtain a positional relationship.

The preset first range may be determined by the control system based on a requirement, a history, a test, and the like, which is not limited in this embodiment. Therefore, the deletion area may be understood as two partial areas in the virtual keyboard, where the deletion key is located in one partial area, that is, the virtual key is located in the virtual keyboard, and another partial area is an area relatively close to the location of the deletion key in the virtual keyboard (that is, the preset first range).

The association relationship may be an association relationship between the upper and lower positions, for example, the first position is located directly above the second position, or the first position is not located directly above the second position.

Accordingly, in the case where the positional relationship characterizes that the first position is located directly above the second position, it is explained that the pinching of the thumb and the index finger is performed directly above the deletion area, that is, the user performs the pinching of the thumb and the index finger directly above the deletion area. In the case where the positional relationship indicates that the first position is not located directly above the second position, it is explained that the pinching of the thumb and the index finger is not performed directly above the deletion area (i.e., the area on the virtual keyboard other than the deletion key and the area where the deletion key presets the first range), i.e., the user performs the pinching of the thumb and the index finger at the position other than directly above the deletion area.

The manner in which the control system obtains the first position and the second position is not limited in this embodiment. Taking the example that the control system obtains the first position, the control system can obtain the first position based on a dynamic following mode, and the specific implementation principle of dynamic following can refer to the related technology, which is not repeated here. Taking the example that the control system obtains the second position, the control system may first obtain the position of the virtual keyboard under the coordinate system where the virtual reality scene is located, and then determine the second position based on the delete key and the position of the preset first range in the virtual keyboard.

It will be appreciated that the description of the position in this embodiment is based on the coordinate system of the virtual reality scene.

S202: and obtaining the action information of the hand of the user after the two fingers are pinched.

For example, in combination with the above analysis, after the user has pinched his thumb and index finger, the user may perform a corresponding operation (e.g., unclamping the pinched thumb and index finger) with respect to the pinch, or perform a corresponding operation (e.g., dragging the pinched thumb and index finger) based on the pinch, where information corresponding to the operation is motion information obtained by the control system.

For example, after pinching the thumb and index finger of the user, the user may release the pinched thumb and index finger, and the motion information includes operation information that the user releases the pinched thumb and index finger. As another example, the user may drag the pinch thumb and index finger after the pinch, and the motion information includes drag information that the user drags the pinch thumb and index finger.

S203: and carrying out corresponding operation on the input text according to the position relation and the action information.

In combination with the above analysis, the user may pinch two fingers at different positions to cause the first position to be different, and the user may perform different operations on the pinched two fingers to cause the action information to be different, so that the control system may perform different operations on the input text for different positional relationships and different action information.

That is, in this embodiment, the control system may perform corresponding operations such as deletion or input on the input text in combination with the positional relationship between the positions of the two fingers and the position of the deletion area and the action information of the two fingers after the two fingers are pinched, so as to improve the diversity and flexibility of the control system for controlling the virtual keyboard, and improve the experience of the user in the virtual reality scene.

In connection with the above analysis, the control system may perform different operations on the input text for different positional relationships and/or motion information.

For example, in one possible implementation, the positional relationship may indicate that the first position is located directly above the second position, that is, the user pinch two fingers directly above the deletion area, the motion information may indicate that the user drags (including up, down, left, and right drags based on the user's viewing angle of the virtual keyboard) the two pinch two fingers after the two fingers pinch, so as to select a part of the text or select all of the text from the input text, and the motion information further indicates that the user releases the two pinch fingers based on the selection.

Accordingly, in the case where the positional relationship indicates that the first position is located directly above the second position, and the action information indicates that at least part of the text in the selected input text is dragged after the two fingers are pinched, and the pinched fingers are released after the at least part of the text is dragged, the control system may perform an operation of deleting at least part of the text. And the control system displays the deleted at least part of the text carrying the deletion identification. For example, the delete is identified as a strikethrough.

In another possible implementation technical solution, the preview box includes a first blinking cursor; the positional relationship may indicate that the first position is not located directly above the second position, i.e., the position of the user directly above the non-deletion area, such as the position directly above the other numeric keys, or the position directly above the first blinking cursor, and the like, and the action information may indicate that the user respectively releases the two fingers of the pinch after the two fingers are pinched twice, i.e., the user two times pinch and release the two fingers at the position directly above the non-deletion area, and if the two fingers are pinched and released once, the situation may be interpreted as a double click of the user at the position directly above the non-deletion area.

Correspondingly, under the condition that the position relation indicates that the first position is not located right above the second position and the action information indicates that the two fingers are loosened after being kneaded twice, the control system can execute deleting of the whole word text in the preset second range of the first flicker cursor. Similarly, the control system may display the whole word text in the preset second range carrying the deletion identifier.

The first blinking cursor refers to a blinking symbol in the preview box, which may be used to indicate a position input by a user or a position of an operation, and the blinking symbol may be a blinking small vertical bar, a small square, a small arrow, or the like. The preset second range may be determined by the control system based on a requirement, a history, and an experiment, which is not limited in this embodiment. Whole word text refers to text that is in units of whole words, such as text that includes one or more phrases.

According to the analysis, the control system may delete at least part of the text and may delete the whole word text within the preset second range according to different position relations and action information, so as to avoid misoperation of erroneous deletion, improve user experience, and expand and display the preview frame under the condition of deletion operation. The control system expands and displays the preview frame under the condition of deleting at least part of the text or deleting the whole word text in a preset second range.

In one possible implementation technical scheme, the control system may also detect the position of the first flicker cursor to monitor, and expand and display the preview frame when detecting the movement of the first flicker cursor.

The expansion display can be that the control system expands the preview frame from the height and/or width so as to expand and display the corresponding text in the preview frame with expanded size; the expansion display can also be that the control system does not adjust the size (such as width and/or height) of the preview frame, but can display the corresponding text outside the preview frame, so as to expand the preview frame by combining the out-of-frame area of the preview frame, and thus expand and display the corresponding text.

That is, the control system may implement different manners of expanding display, and correspondingly, in a possible implementation technical solution, the control system may pre-configure an initial manner of expanding display, and the user may switch different manners of expanding display based on touch operation, for example, the control system outputs different manners of expanding display when detecting a user switching request, and uses, according to a selected operation of the user, a manner of expanding display corresponding to the selected operation as a manner of expanding display in a current interaction scene.

Or the control system can output various expansion display modes and acquire the selected operation of the user, so that the expansion display mode corresponding to the selected operation is used as the expansion display mode in the current interaction scene.

Correspondingly, taking the example that the control system combines the out-of-frame area of the preview frame to expand the preview frame, the text can be displayed outside the preview frame, and in particular, a mode of gradual change of font clarity is adopted, for example, the higher the clarity of the text which is closer to the preview frame.

In another possible implementation solution, as shown in fig. 3, the preview box includes two partial areas, where one partial area is an area for performing output display on the input text (such as a text display area shown in fig. 3), and the other partial area is a deletion operation area. The control system can delete the input text based on the click operation of the user in the click deletion operation area.

For example, if the user clicks once in the delete operation region, the control system deletes the previous character adjacent to the first blinking cursor based on the position of the first blinking cursor; if the user presses the deletion operation area for a long time, the control system sequentially deletes the characters adjacent to the first flicker cursor based on the position of the first flicker cursor until the user releases the touch operation on the deletion operation area or until the input text is deleted completely.

In combination with the above analysis, the preview box includes the first flicker cursor, so as to improve the flexibility of controlling the first flicker cursor, facilitate the user's experience of operating the first flicker cursor, and in a possible implementation technical solution, the control system may control the movement of the first flicker cursor based on the positional relationship and the action information.

For example, in a case where the positional relationship indicates that the first position is not located directly above the second position, and the motion information indicates that the two finger pinch-and-drag is released after the third position, the control system controls the first blinking cursor to move based on the two finger pinch-and-drag until the third position corresponds to the position in the preview frame.

That is, in the case where the control system monitors that the two fingers of the user pinch over the non-deletion area and drags the pinched two fingers until the third position, the control system controls the first blinking cursor to move from its original position to the third position corresponding to the position in the preview frame.

For example, as shown in fig. 4 in combination with fig. 4 and 5, the "delete key" in fig. 4 is a partial region in the "delete region" (it should be understood that the "delete region" and the "delete key" in fig. 4 are merely exemplary to illustrate the possible spatial relationship of the delete region and the delete key, and cannot be understood as spatial limitation of the two), and as can be seen from fig. 4, the thumb and the index finger of the user are kneaded in a region other than the delete region, and as can be seen from fig. 5, in a case where the user releases the thumb and the index finger of the dragging kneading from the start position of kneading in the third position, the control system controls the first blinking cursor to move to a position corresponding to the third position in the preview frame.

In one possible implementation, if the control system monitors that the user is pinching two fingers at the position of the first blinking cursor in the preview pane and drags the pinched fingers to move to the target position in the preview pane, the control system controls the first blinking cursor to move from the original position (i.e., the position where the user is pinching two fingers) to the target position.

That is, in the case where the positional relationship indicates that the first position is not located directly above the second position and the first position is located directly above the first blinking cursor, and the motion information indicates that the pinching and dragging of the two fingers to the target position is released, the control system controls the first blinking cursor to move based on the dragging of the two pinched fingers until moving to the target position.

Correspondingly, in one possible implementation technical solution, the virtual reality scene further includes a second blinking cursor outside the preview frame, and when the control system controls the movement of the first blinking cursor, the control system can synchronously control the movement of the second blinking cursor, that is, the control system can control the movement of the second blinking cursor to follow the movement of the first blinking cursor.

It can be understood that, either the physical keyboard is operated by the user or the virtual keyboard is operated by the user, or there may be a situation that the user needs to modify the information input based on the operation, so as to improve the convenience of the user in modifying the information already input, so as to improve the reliability of information input, the control system may perform corresponding operations based on the cancel key and/or the redo key in the virtual keyboard by the user.

The virtual keyboard also illustratively includes a dismiss key and/or a redo key. The cancel key is used for performing cancel processing on the previous operation. The redo key is used for carrying out the redo processing on the previous operation, or the cancel key and the redo key are used for carrying out the redo processing on the previous operation.

The previous operation may be understood as an operation performed by the user based on the virtual keyboard and adjacent to the current operation, such as an operation of inputting text by the user, an operation of adjusting a color of the text by the user, an operation of editing a picture by the user, and the like, before the current operation.

That is, a cancel key may be included in the virtual keyboard, taking as an example text that was previously entered, and the control system may cancel text that has been entered based on the user's operation of the cancel key. The virtual keyboard may include a redo key therein, and the control system may redo text that has been entered based on user manipulation of the redo key. The virtual keyboard may include a cancel key and a redo key therein, and the control system may redo text that has been entered based on user manipulation of the cancel key and the redo key.

Illustratively, taking the example that a dismissal key is included in the virtual keyboard, and the dismissal key is used to dismiss the input text, the control system may dismiss the input text if the user clicks the dismissal key or two fingers pinch and then release the input text.

For example, referring to fig. 6 and fig. 7, as shown in fig. 6, the virtual keyboard includes a preview box and further includes a dismissal key, as shown in fig. 7, a user may click on the dismissal key, that is, the user may click on the dismissal key, and accordingly, in case that the user clicks on the dismissal key, the control system dismisses the text that has been input. Alternatively, the user may pinch two fingers and release them just above the cancel key (not shown in the figure), in which case the control system may also control the cancel of the text that has been entered.

On the other hand, in the virtual reality scenario, especially in the VR scenario and the AR scenario, it is difficult for the user to accurately locate the positions of the virtual keys in the virtual keyboard, so that the user often needs to look at the virtual keyboard input content, and the input content is output and displayed through the preview box. However, the preview box is often located at a certain distance from the virtual keyboard, and the user's line of sight is often repeatedly switched between the preview box and the virtual keyboard during input. In this embodiment, as shown in fig. 6, the preview frame is located above the virtual keyboard, and the distance between the preview frame and the virtual keyboard is relatively short, so as to reduce the operation burden caused by repeated switching of the user's line of sight.

On the other hand, in a virtual reality scenario, particularly in an MR scenario, the distance between the preview box and the user is large, and it is often difficult for the user to locate a specific character for modification. By combining the indicator of the flashing cursor in the form of spaced kneading and dragging, the user can more easily position the position to be modified so as to perform operations such as insertion or deletion at the corresponding position. Meanwhile, the movement attribute of the space kneading and dragging is linked with the deleting key, so that the continuous deleting function is realized, and the use efficiency is improved while the psychological expectation of a user is met.

Taking the example that the virtual keyboard comprises a redo key, the redo key is used for redoing the input text, the control system can redo the input text under the condition that the click operation of the user on the redo key or the two fingers are loosened after being pinched.

For example, the virtual keyboard includes a preview box and a redo key, and the user can click the redo key, that is, the user can click the redo key, and accordingly, in the case that the user clicks the redo key, the control system can control the redo of the text that has been input. Alternatively, the user may pinch two fingers directly over the redo key and unclamp, in which case the control system may also control the redo of the text that has been entered.

Taking the example that the virtual keyboard comprises a dismissal key and a redo key, wherein the dismissal key and the redo key are used for redoing the input text, and under the condition that two fingers are kneaded right above the dismissal key and are loosened after dragging a preset distance to the direction of the redo key, the control system controls the redo input text.

Similarly, the preset distance may be determined by the control system based on the requirement, the history, and the test, which is not limited in this embodiment. For example, the position corresponding to the preset distance may be a position between the cancel key and the redo key; for another example, the position corresponding to the preset distance may be a position coinciding with the redo key; for another example, the position corresponding to the preset distance may be a position of the secondary key away from the withdrawal key.

Illustratively, referring to fig. 8 and 9, as shown in fig. 8, the virtual keyboard includes a dismissal key and a redo key thereon, and the dismissal key and the redo key are located at the upper left region of the virtual keyboard, at the left side of the preview box. The user may pinch two fingers directly above the undo key, as illustrated in fig. 8, and the user may pinch the thumb and index finger directly above the undo key, and as illustrated in fig. 8, the undo key is located at the left side of the redo key, so the user may continue to drag the pinched thumb and index finger with the thumb and index finger pinched directly above the undo key, as illustrated in fig. 9, until dragging to the thumb and index finger directly above the redo key releases the pinch, in which case the control system controls the redo to enter text.

The directions of the upper, left, upper left, etc. in this embodiment are based on the viewing angle of the user viewing the virtual keyboard.

In one possible technical scheme, the control system does not output and display the cancel key and the redo key under the condition that text content is not in the preview box; when the preview frame comprises text content, the control system can output and display the cancel key first, and control the output and display of the redo key under the condition that the user is monitored to pinch two fingers right above the cancel key; alternatively, the control system may output and display both the dismissal key and the redo key.

Taking the control system to output and display the cancel key and the redo key successively as an example, under the condition that the user clicks the cancel key, outputting the redo key in a preset third range of the cancel key, and under the condition that two fingers are kneaded and dragged to the redo key right above the cancel key and then released, controlling the redo of the input text by the control system. Or,

the control system firstly outputs and displays the cancel key, and when a user kneads two fingers right above the cancel key, the control system outputs and displays the redo key, and when the two kneaded fingers drag to right above the redo key to loosen, the control system controls the redo of the input text.

The analysis above mainly describes the redo of the input text by the control system in combination with the dimensions of the dismiss key and the redo key from the kneading, and in another possible implementation solution, the control system may redo the input text by combining the dismiss key and the redo key from the dimensions of clicking.

For example, in the case where the control system outputs the cancel key and the redo key simultaneously, if the user clicks the cancel key directly above the cancel key by a finger and drags to directly above the redo key, and then removes the finger from directly above the redo key, the control system controls the redo of the input text.

For another example, in the case that the control system outputs the cancel key first and then outputs the redo key, if the user clicks the cancel key directly above the cancel key by a finger, the control system outputs the redo key, and if the user drags the finger directly above the redo key and then removes the finger from directly above the redo key, the control system controls the redo of the input text.

It should be noted that, in the case that the control system uses the kneading method to undo and redo the input text, the interaction method in the virtual reality scene (especially the MR scene) is more adapted than the method of "clicking the key/pressing the combined shortcut" in the related art. The method can greatly lighten the burden of the user in the aspect of operation precision, namely the user does not need to control the fingertips of the hands to click a small-range area. For MR scenes, the tactile feedback is relatively absent, and the accuracy and the sensitivity of the user for controlling the finger suspension operation are difficult to keep consistent level when interacting with the entity, so that the requirement on the operation accuracy of the user for driving the finger to move by using wrist/arm strength is greatly reduced. Meanwhile, the cancel and the redo of the input text are also common functions in the keyboard input of the virtual keyboard, and the memory burden of a user can be greatly reduced by placing the trigger areas of the two functions on the side surface (the upper left corner as shown in fig. 8) of the virtual keyboard.

In one possible technical scheme, the virtual keyboard may not include a cancel key and a redo key, the virtual keyboard may include a vertical bar, and the control system controls the redo of the previous operation under the condition that two fingers of the user pinch the vertical bar to swing outwards; in the case where the user's two fingers pinch the vertical bar to swing inward, the control system controls the withdrawal of the previous operation.

Alternatively, one vertical bar may be two vertical bars, one for undoing a previous operation and the other for redoing a previous operation.

For example, for ease of distinction, we refer to two vertical bars as a first vertical bar and a second vertical bar, respectively. Taking the first vertical bar as an example for canceling an already entered text, the control system controls the canceling of the entered text in case the user clicks on the first vertical bar or the user pinches the first vertical bar with two fingers and unclamps. Accordingly, taking the second vertical bar for the example of the redo of the text that has been entered, the control system controls the redo of the entered text in the event that the user clicks on the second vertical bar, or the user pinches the second vertical bar with two fingers and unclamps.

The principle of the control system for cancelling and redoing the input text can be applied to the function of copying and pasting the input text.

For example, taking the first vertical bar as an example for copying text that has been entered, the control system controls copying of the entered text in the event that the user clicks on the first vertical bar, or the user pinches the first vertical bar with two fingers and unclamps. Accordingly, taking the case that the second vertical bar is used for pasting the copied text as an example, the control system controls the pasting of the copied text in the case that the user clicks the second vertical bar or the user kneads the second vertical bar with two fingers and releases the second vertical bar.

In another possible implementation solution, as shown in fig. 8, the virtual keyboard may include a dismissal key and a redo key, where the control system obtains a residence time of the user's hand on the dismissal key if detecting that the user's hand clicks the dismissal key, and if the residence time is less than a preset time threshold, it indicates that the user clicks the dismissal key, the control system performs a dismissal operation on the previous operation; otherwise, if the dwell time reaches the preset threshold value and the control system detects that the hand of the user moves rightward after clicking the cancel key (the redo key is displayed on the right side of the cancel key as shown in fig. 8), the control system judges the size between the distance dragged by the hand of the user and the preset distance threshold value, and if the distance dragged by the hand of the user reaches the preset distance threshold value, the control system executes the redo operation on the previous operation.

In order to improve the operation perception of the user, the control system can display a dragging progress bar so as to display the distance between the current position of the user hand and a preset distance threshold in a progress bar mode, so that the experience of the user on the virtual keyboard is improved.

In one possible implementation technical solution, the control system may adjust related attributes of the virtual keyboard based on the operation of the user, so as to improve flexibility and diversity of control of the virtual keyboard. Wherein, the related attributes of the virtual keyboard may include one or more of an angle of the virtual keyboard, a position of the virtual keyboard, closing of the virtual keyboard, and the like.

For example, two adjustment bars may be included on the virtual keyboard, for convenience of distinction, we may refer to the two adjustment bars as a first adjustment bar and a second adjustment bar, the control system may adjust the angle of the virtual keyboard based on the first adjustment bar, the control system may adjust the position of the virtual keyboard based on the second adjustment bar, and the control system may close the virtual keyboard based on the first adjustment bar or the second adjustment bar.

That is, different adjustment bars may be included on the virtual keyboard, and the control system implements control of different attributes of the virtual keyboard based on the different adjustment bars.

Taking the example that the first adjusting lever is used for adjusting the angle of the virtual keyboard, under the condition that the first adjusting lever is selected by kneading two fingers, the control system can control the virtual keyboard to change along with the change of the first adjusting lever kneaded by two fingers in the vertical axis direction (Z axis) so as to adjust the angle of the virtual keyboard.

The adjusting bars can be a transverse bar or a vertical bar, and correspondingly, the two adjusting bars can comprise two transverse bars, two vertical bars and one transverse bar and one vertical bar.

For example, two adjustment bars are taken as two bars, such as a first adjustment bar is taken as a first bar, a second adjustment bar is taken as a second bar, as shown in fig. 6, a bar is included in the virtual keyboard, and fig. 6 is exemplarily shown by taking a virtual keyboard including one bar as an example. If the bar is the first bar, the angle of the virtual keyboard may be adjusted in the vertical axis direction in the case where the user kneads the bar, and the effect of adjusting the front and rear may be referred to as fig. 10.

The angle in this embodiment is based on the coordinate system of the virtual reality scene where the virtual keyboard is located.

In one possible technical scheme, the control system may preset an adjustment gear of the angle, and construct a corresponding relationship between the operation of the user kneading the first bar and the adjustment gear, so as to determine the adjustment gear corresponding to the operation of the user kneading the first bar, thereby adjusting the current angle of the virtual keyboard based on the adjustment gear.

Similarly, the number of the adjustment gears may be two or more, and may be determined by the control system based on the requirements, the history, the experiments, and the like, which is not limited in this embodiment.

Taking the example that the second bar is used for adjusting the position of the virtual keyboard, under the condition that the second bar is selected by kneading two fingers, the control system can control the virtual keyboard to change along with the change of the second bar in the translational direction by kneading two fingers in the translational direction, and the translational direction comprises a horizontal axis (X axis) translational direction and/or a vertical axis (Y axis) translational direction.

Similarly, if the bar shown in fig. 6 is the second bar, the position of the virtual keyboard may be adjusted in the horizontal axis direction and the vertical axis direction, that is, the front, rear, left, and right positions of the virtual keyboard in the virtual reality scene coordinate system, when the user pinches the bar.

Taking the case that the first bar or the second bar is used for closing the virtual keyboard as an example, when two fingers pinch the first bar or the second bar and the two pinched fingers are released under the preset condition, the control system controls the virtual keyboard to be closed. The preset conditions comprise:

the speeds of the two fingers in the vertical axis direction reach a preset first threshold. Or,

Similarly, the first threshold and the second threshold may be determined by the control system based on a requirement, a history, and an experiment, which is not limited in this embodiment.

There may be two cases when the control system controls the virtual keyboard to be closed, taking the bar for closing the virtual keyboard as the first bar as an example, one case is: after the user kneads the first bar, the arm of the hand of the user kneading the first bar can drop freely, if the speed of the dropping operation of the user in the vertical axis direction is high, if a preset first threshold value is reached, the control system controls the virtual keyboard to be closed under the condition that the user loosens the kneaded two fingers.

Another case is: after the user kneads the first bar, if the control system determines that the position of the virtual keyboard in the longitudinal axis direction is smaller than a preset second threshold value, the control system controls the virtual keyboard to be closed under the condition that the user loosens the kneaded two fingers. Alternatively, the case may be: after the user pinches the first bar, if the control system determines that the distance between the pinched two fingers and the virtual keyboard in the longitudinal axis direction is smaller than a preset second threshold value, the control system controls the virtual keyboard to be closed under the condition that the user loosens the pinched two fingers.

In one possible implementation solution, in order to facilitate the user to distinguish between the bars used for adjusting the angle and the position of the virtual keyboard, that is, in order to facilitate the user to distinguish between the first bar and the second bar, so that the user performs an operation on the corresponding bars based on the needs thereof, the first bar and the second bar output by the control system are different in size of area, for example, the area of the first bar is larger than the area of the second bar, or the area of the first bar is smaller than the area of the first bar.

It should be noted that, in the related art, the position and angle of the virtual keyboard are fixed, and the fixed position and angle are difficult for the user to make changes when the user needs to adjust, and often the user adapts to the virtual keyboard. However, in the present embodiment, the control system adjusts the related attribute of the virtual keyboard in the above-described manner, such as the adjustment of the angle and the adjustment of the position by the control system described in the above-described example.

Correspondingly, in the case that the adjusting bar is a vertical bar, and in the case that one of the two adjusting bars is a horizontal bar, and one is a vertical bar, for example, the first adjusting bar is a horizontal bar, and the second adjusting bar is a vertical bar, the execution logic of the control system may also refer to the implementation principle that the adjusting bar is a horizontal bar in the above example, which is not described herein again.

In one possible implementation, the virtual keyboard includes a button thereon, and the control system may close the virtual keyboard based on the button.

For example, the control system may monitor the touch operation of the user, and in case that the control system monitors that the user clicks the button, the control system may control the virtual keyboard to be closed. In this embodiment, the attributes (such as shape attribute and color attribute) of the button are not limited, and may be determined by the control system based on requirements, history, and the like, or may be determined by the control system based on interaction between the control system and the user.

It can be understood that selection and switching of the input method are important contents in the virtual keyboard, so as to facilitate the user to switch between different input methods, and improve flexibility and reliability of control of the control system on the dimension of the input method in the control system.

The virtual keyboard further comprises an input method switching key, and the control system switches the current input method of the virtual keyboard to a preset input method under the condition that the input method switching key is clicked by a finger of a user, and the speed of the input method switching key is changed from a speed value smaller than a preset third threshold value to a speed value larger than the preset third threshold value. And/or the number of the groups of groups,

When the time length of the finger touch input method switching key of the user reaches a preset fourth threshold value, or the speed of the finger touch input method switching key of the user is smaller than a preset fifth threshold value, and the acceleration of the finger touch input method switching key of the user changes from an acceleration value larger than a preset sixth threshold value to an acceleration value smaller than the preset sixth threshold value, and when a preset cancel condition is not met within the preset time length, or the speed of the finger touch input method switching key of the user changes from a speed value larger than a preset seventh threshold value to a speed value smaller than the preset seventh threshold value, and when the cancel condition is not met within the preset time length, the control system outputs a secondary menu comprising a plurality of input methods, wherein the preset cancel condition is that the speed of the finger touch input method switching key of the user changes from a speed value smaller than a preset eighth threshold value to a speed value larger than the preset eighth threshold value, and the secondary menu is used for determining an input method switched by the current input method based on clicking operation of the secondary menu by the user.

Similarly, the third threshold, the fourth threshold, the fifth threshold, the sixth threshold, the seventh threshold, the eighth threshold, and the preset duration may be determined by the control system based on the requirements, the history, and the experiment, which is not limited in this embodiment.

As shown in fig. 11, the virtual keyboard further includes an input method switching key disposed at the lower left corner of the virtual keyboard, and the user may click the input method switching key by a finger, for example, the speed at which the user clicks the input method switching key jumps from a speed value smaller than a preset third threshold to a speed value greater than the third threshold, that is, the user clicks the input method switching key once by a finger, in which case, the control system may switch the current input method to the preset input method based on the clicking operation of the user.

The preset input method may be preset by the control system based on a setting operation of the user in advance, may be preset by the control system based on a content of a history selection operation of the user in the input method, and the like, and the embodiment is not limited.

For example, the preset input method includes two input methods commonly used by a user, and for convenience of distinction, one of the input methods may be referred to as a first input method, and the other input method may be referred to as a second input method. If the current input method is the first input method, the control system switches the first input method to the second input method under the condition that the user clicks the input method switching key quickly by the finger.

The above example is a control principle of the control system under the condition that the user clicks the input method switching key quickly, if the user does not click the input method switching key quickly, such as the user presses the input method switching key for a long time, the control system may output a secondary menu including a plurality of input methods, so that the user selects the input method that the user desires to use, thereby switching the current input method to the input method selected by the user.

For example, as shown in fig. 11, fig. 12, and fig. 13, if the control system monitors that the finger of the user presses the input method switching key for a long time, and if the time length of the finger touching the input method switching key is greater than or equal to the fourth threshold, the control system determines that the user has an intention to select the input method that the user desires to use from the input methods, and the control system outputs a secondary menu as shown in fig. 12, where the secondary menu includes multiple input methods for the user to select from the multiple input methods. As shown in fig. 13, the user may select the desired input method through the operation of clicking with a finger, and accordingly, the control system switches the current input method to the input method clicked by the user based on the clicking operation of the user.

Alternatively, the control system may determine whether to output the secondary menu based on the monitored information about the acceleration and/or speed of the user clicking the input method switching key.

For example, when the control system monitors that the speed of the user's finger touch input method switching key is less than a preset fifth threshold value, the acceleration of the user's finger touch input method switching key jumps from an acceleration value greater than a preset sixth threshold value to an acceleration value less than the preset sixth threshold value, and the speed of the user's finger touch input method switching key does not jump from a speed value less than the preset eighth threshold value to a speed value greater than the preset eighth threshold value within a preset time period, the control system outputs a secondary menu.

The condition that the speed of the user's finger touch input method switching key is smaller than a preset fifth threshold value and the acceleration of the user's finger touch input method switching key jumps from an acceleration value larger than a preset sixth threshold value to an acceleration value smaller than the preset sixth threshold value is understood as that the control system determines to enter a preparation stage of outputting the secondary menu, the preset duration is understood as a time length of the preparation stage, if the control system has entered the preparation stage (i.e. the above condition is satisfied) and the control system does not monitor that the speed of the user's finger touch input method switching key does not jump from a speed value smaller than the preset eighth threshold value to a speed value larger than the preset eighth threshold value within the time length, the control system determines to enter the output stage, i.e. to output the secondary menu, otherwise, if the control system does not monitor that the speed of the user's finger touch input method switching key does not jump from a speed value smaller than the preset eighth threshold value to a speed value larger than the preset eighth threshold value within the time length, the control system determines not to enter the output stage, i.e. not output the secondary menu.

For another example, when the control system monitors that the speed of the user's finger touch input method switching key jumps from a speed value greater than a preset seventh threshold to a speed value less than the preset seventh threshold, and the speed of the user's finger touch input method switching key does not jump from a speed value less than the preset eighth threshold to a speed value greater than the preset eighth threshold within a preset duration, the control system outputs the secondary menu.

Similarly, the condition that the speed of the finger touch input method switching key of the user jumps from the speed value larger than the preset seventh threshold value to the speed value smaller than the preset seventh threshold value may be understood as that the control system determines to enter the preparation stage of outputting the secondary menu, the preset duration may be understood as the time length of the preparation stage, if the control system does not monitor the condition that the speed of the finger touch input method switching key of the user does not jump from the speed value smaller than the preset eighth threshold value to the speed value larger than the preset eighth threshold value within the time length, the control system determines to enter the output stage, that is, output the secondary menu, otherwise, if the control system does not monitor the condition that the speed of the finger touch input method switching key of the user does not jump from the speed value smaller than the preset eighth threshold value to the speed value larger than the preset eighth threshold value within the time length, the control system determines not enter the output stage, that is not output the secondary menu.

It should be understood that the foregoing examples including the different "preparation conditions" may be implemented by the control system alone, or may be combined by the control system to form an embodiment, which is not limited thereto. For example, the control system may combine the various preparation conditions described above to arrive at and implement a new embodiment.

Similarly, the switching of the input method is also a function commonly used in the keyboard input of the virtual keyboard, and the memory burden of the user can be greatly reduced by placing the triggering area of the function on the side surface (the lower left corner as shown in fig. 11) of the virtual keyboard.

It should be noted that, in this embodiment, the control of the control system on the virtual keyboard is described from multiple dimensions, such as a dimension of text processing, a dimension of virtual keyboard adjustment, a dimension of input method adjustment, etc., however, the control of each dimension by the control system may be implemented separately, may be implemented partially, or may be implemented in whole, and cannot be understood as the necessity of implementing each dimension together.

Based on the above technical concept, the present disclosure further provides a processor-readable storage medium storing a computer program for causing the processor to execute the control method of the virtual keyboard according to any one of the embodiments.

According to the technical concept described above, the present disclosure further provides a computer program product, including a computer program, which when executed by a processor implements the control method of the virtual keyboard according to any one of the embodiments described above.

According to the technical concept, the present disclosure further provides a control system of a virtual keyboard, including:

at least one processor in communication with the at least one memory;

wherein the method for controlling a virtual keyboard as described in any of the embodiments above is implemented when the at least one processor executes the at least one set of instructions.

According to the technical concept, the present disclosure further provides an electronic device, including: a processor, and a memory communicatively coupled to the processor;

the memory stores computer-executable instructions;

the processor executes the computer-executable instructions stored in the memory to implement the method for controlling a virtual keyboard as described in any of the embodiments above.

Fig. 14 is a hardware configuration diagram of an electronic device 1400 according to an embodiment of the disclosure. The electronic device 1400 may perform the control method of the virtual keyboard as described in any of the embodiments above.

Taking an example that the control method of the virtual keyboard in the embodiment of the present disclosure may be applied to an application scenario as shown in fig. 1, the following description is given to the electronic device 1400:

in the case where the control method of the virtual keyboard described in any of the above embodiments is executed on the virtual reality apparatus 102, the electronic device 1400 may be the virtual reality apparatus 102. In the case where the control method of the virtual keyboard as described in any of the above embodiments is executed on the server 103, the electronic device 1400 may be the server 103. In the case where the control method of the virtual keyboard according to any of the above embodiments is partially executed on the virtual reality device 102 and partially executed on the server 103, the electronic apparatus 1400 may include the virtual reality device 102 and the server 103.

As shown in fig. 14, the electronic device 1400 may include at least one storage medium 1401 and at least one processor 1402. In some embodiments, electronic device 1400 may also include a communication port 1403 and an internal communication bus 1404. Meanwhile, the electronic device 1400 may further include an Input/Output (I/O) component 1405.

The internal communication bus 1404 may connect the various system components including the storage medium 1401, the processor 1402, and the communication ports 1403. The I/O component 1405 supports input/output between the electronic device 1400 and other components. Communication port 1403 is for communication of data between electronic device 1400 and the outside world, e.g., communication port 1403 may be for communication of data between electronic device 1400 and network 104. Communication port 1403 may be a wired communication port or a wireless communication port.

The storage medium 1401 may include a data storage device. The data storage device may be a non-transitory storage medium or a transitory storage medium. For example, the data storage device may include one or more of a magnetic disk 14011, a Read-Only Memory (ROM) 14012, or a random access Memory (Random Access Memory, RAM) 14013. The storage medium 1401 also includes at least one set of instructions stored in the data storage device. The instructions are computer program code that may include programs, routines, objects, components, data structures, procedures, modules, etc. that perform the control methods of the virtual keyboards provided herein.

The at least one processor 1402 may be communicatively coupled with at least one storage medium 1401 and a communication port 1403 via an internal communication bus 1404. The at least one processor 1402 is configured to execute the at least one instruction set described above. When the electronic device 1400 is running, the at least one processor 1402 reads the at least one instruction set and performs the control method of the virtual keyboard provided herein according to an instruction of the at least one instruction set. The processor 1402 may perform all the steps involved in the control method of the virtual keyboard. The processor 1402 may be in the form of one or more processors, in some embodiments the processor 1402 may include one or more hardware processors, such as microcontrollers, microprocessors, reduced instruction set computers (Reduced Instruction Set Computer, RISC), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), application specific instruction set processors (Application Specific Instruction Processor, ASIP), central processing units (Central Processing Unit, CPU), graphics processing units (graphics processing unit, GPU), physical processing units (Physics Processing Unit, PPU), microcontroller units, digital signal processors (Digital Signal Processor, DSP), field programmable gate arrays (Field Programmable Gate Array, FPGA), advanced RISC Machines (ARM), programmable logic devices (Programmable Logic Device, PLD), any circuit or processor capable of performing one or more functions, or the like, or any combination thereof. For illustrative purposes only, only one processor 1402 is depicted in the electronic device 1400 in this specification. However, it should be noted that the electronic device 1400 may also include multiple processors in the present specification, and thus, the operations and/or method steps disclosed in the present specification may be performed by one processor as described in the present specification, or may be performed by a combination of multiple processors. For example, if the processor 1402 of the electronic device 1400 performs steps a and B in this specification, it should be understood that steps a and B may also be performed by two different processors 1402 in combination or separately (e.g., a first processor performs step a, a second processor performs step B, or the first and second processors perform steps a and B together).

It will be apparent to those skilled in the art that embodiments of the present disclosure may be provided as a method, apparatus, system, or computer program product. Accordingly, the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present disclosure may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, magnetic disk storage, optical storage, and the like) having computer-usable program code embodied therein.

The present disclosure is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the disclosure. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer-executable instructions. These computer-executable instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These processor-executable instructions may also be stored in a processor-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the processor-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These processor-executable instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

It will be apparent to those skilled in the art that various modifications and variations can be made to the present disclosure without departing from the spirit or scope of the disclosure. Thus, the present disclosure is intended to include such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.

Claims

1. The control method of the virtual keyboard is characterized by being applied to a virtual reality scene, wherein the virtual reality scene comprises a virtual keyboard, a user and a preview frame, the preview frame comprises a first flicker cursor, the preview frame comprises an input text input by the user based on the virtual keyboard, the virtual keyboard comprises a delete key, and the hand of the user is positioned right above the virtual keyboard and does not touch the virtual keyboard; the method comprises the following steps:

obtaining motion information of the hand of the user after the two fingers are pinched, wherein the motion information comprises that the user loosens the pinched two fingers, and then, the two fingers are pinched again and loosened; and

and deleting the whole word text in a preset second range of the first flickering cursor under the condition that the position relation indicates that the first position is not located right above the second position and the action information indicates that the user loosens the two pinched fingers and then loosens the two fingers again.

2. The method according to claim 1, wherein the method further comprises:

deleting at least part of the text to be deleted and displaying the deleted at least part of the text carrying a deletion mark under the condition that the position relation represents that the first position is located right above the second position, and the action information represents that at least part of the text in the input text is dragged after the two fingers are kneaded, and the kneaded fingers are released after the at least part of the text is dragged; and/or the number of the groups of groups,

and displaying the whole word text in the preset second range carrying the deletion mark.

3. The method according to claim 2, wherein the method further comprises:

4. The method of claim 1, wherein the preview box includes a first blinking cursor therein, and the virtual reality scene further includes a second blinking cursor outside of the preview box; the method further comprises the steps of:

5. The method of claim 1, wherein the virtual keyboard further comprises a dismissal key and/or a redo key; and

the redo key is used for carrying out redo processing on the previous operation, or the undo key and the redo key are used for carrying out redo processing on the previous operation;

the revocation key is used for performing revocation processing on a previous operation, and comprises the following steps:

in the case of obtaining the click operation of the user on the cancel key or the release after the two fingers are pinched, performing cancel processing on the previous operation;

the redo key is used for carrying out redo processing on the previous operation, and comprises the following steps:

6. The method of claim 5, wherein the dismiss key and the redo key are used to redo a previous operation, comprising:

7. The method of claim 1, wherein the virtual keyboard includes an adjustment bar thereon, the adjustment bar being a cross bar and/or a vertical bar; the adjusting bars comprise a first adjusting bar and a second adjusting bar; and

the first adjusting lever or the second adjusting lever is used for closing the virtual keyboard;

the first adjusting lever is used for adjusting the angle of the virtual keyboard, and comprises: controlling the virtual keyboard to change along the change of the two fingers kneading the first adjusting bar in the vertical axis direction under the condition that the two fingers kneading the first adjusting bar;

8. The method of claim 7, wherein the first adjustment bar or the second adjustment bar is used to close the virtual keyboard, comprising:

wherein, the preset conditions include:

9. The method of claim 1, wherein the virtual keyboard further comprises an input method switch key; the method further comprises the steps of:

10. A control system for a virtual keyboard, comprising:

at least one processor in communication with the at least one memory;

wherein the method of any of claims 1 to 9 is implemented when the at least one processor executes the at least one set of instructions.