CN115421646A

CN115421646A - Virtual keyboard processing method and device, electronic equipment, storage medium and product

Info

Publication number: CN115421646A
Application number: CN202211043316.XA
Authority: CN
Inventors: 楼瀚琼; 顾友达; 汪世涛; 葛鹏
Original assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Current assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Priority date: 2022-08-29
Filing date: 2022-08-29
Publication date: 2022-12-02

Abstract

The invention provides a virtual keyboard processing method, a virtual keyboard processing device, electronic equipment, a storage medium and a product, and relates to the technical field of interface traversal tests, in particular to the field of keyboard scene processing in the traversal test process. The specific implementation scheme is as follows: in the traversal test process of the application interface control function, carrying out state detection on a virtual keyboard in the application interface control; in response to detecting that the virtual keyboard is in an open state, determining a consecutive number of times that the virtual keyboard is continuously in an open state; and processing the virtual keyboard according to the continuous times. By the method and the device, the keyboard control depending on the virtual keyboard and the event related to the keyboard control can be tested, and the phenomenon that the traversing efficiency is influenced by the fact that the keyboard is covered for a long time is avoided.

Description

Virtual keyboard processing method and device, electronic equipment, storage medium and product

Technical Field

The present disclosure relates to the field of interface traversal testing technologies, and in particular, to the field of keyboard scene processing in a traversal testing process, and in particular, to a virtual keyboard processing method and apparatus, an electronic device, a storage medium, and a product.

Background

In the process of traversing test of an application interface (UI) of the mobile terminal, operations related to virtual keyboard controls may occur, the functions actually covered by the operations are substantially the same, and other controls can be covered for a long time to influence the traversing effect.

Forcing the keyboard cover out results in some of the controls and triggered events that are only revealed when the keyboard is turned up being untested.

Disclosure of Invention

The disclosure provides a virtual keyboard processing method, a virtual keyboard processing device, electronic equipment, a storage medium and a product.

According to a first aspect of the present disclosure, there is provided a virtual keyboard processing method, the method including:

in the traversal test process of the application interface control function, carrying out state detection on a virtual keyboard in the application interface control; determining the continuous times of the virtual keyboard continuously in the keyboard scene in response to detecting that the virtual keyboard is in the open state; and processing the virtual keyboard according to the continuous times.

According to a second aspect of the present disclosure, there is provided a virtual keyboard processing apparatus, the apparatus comprising:

the detection module is used for carrying out scene detection on the virtual keyboard in the application interface control in the traversal test process of the application interface control function; the determining module is used for responding to the detection that the virtual keyboard is in the opening state, and determining the continuous times of the virtual keyboard in the opening state continuously; and the processing module is used for processing the virtual keyboard according to the continuous times.

According to a third aspect of the present disclosure, there is provided an electronic device comprising:

at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of the first aspect.

According to a fourth aspect of the present disclosure, there is provided a non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method according to the first aspect.

According to a fifth aspect of the present disclosure, there is provided a computer product comprising a computer program which, when executed by a processor, implements the method according to the first aspect.

It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.

Drawings

The drawings are included to provide a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:

FIG. 1 is a schematic diagram of an application environment according to an embodiment of the present disclosure;

fig. 2 is a schematic flowchart illustrating a virtual keyboard processing method according to an embodiment of the present disclosure;

FIG. 3 is a flowchart illustrating a keyboard processing method based on click rate according to an embodiment of the present disclosure;

FIG. 4 is a flowchart illustrating a method for adjusting a click rate of a keyboard control according to an embodiment of the present disclosure;

fig. 5 illustrates a flowchart of a method for canceling a virtual keyboard based on a keyboard control according to an embodiment of the present disclosure;

fig. 6 is a flowchart illustrating a keyboard control obtaining method according to an embodiment of the present disclosure;

FIG. 7 is a general diagram illustrating a method for canceling a virtual keyboard according to an embodiment of the disclosure;

fig. 8 shows a schematic structural diagram of a virtual keyboard processing apparatus provided in an embodiment of the present disclosure;

FIG. 9 illustrates a schematic block diagram of an example electronic device 900 that can be used to implement embodiments of the present disclosure.

Detailed Description

Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, in which various details of the embodiments of the disclosure are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

In the process of traversing and testing the application interface of the mobile terminal, operations related to the virtual keyboard control may occur, the functions actually covered by the operations are substantially the same, and other controls can be covered for a long time to influence the traversing effect.

In the related art, there are two main ways for the virtual keyboard scene processing scheme in the mobile terminal UI traversal test. One is to force the elimination of the virtual keyboard impact. The specific scheme is that whether the traversal is in the keyboard scene is identified based on keyword (such as XCUIElementTypeKeyboard) matching of the virtual keyboard in the control tree, and the virtual keyboard covering is eliminated through a sliding-down operation of wda when the traversal is identified as the keyboard scene. When the virtual keyboard is not folded, the virtual keyboard control is filtered to avoid clicking the keyboard control, and the input is completed directly through the input operation of wda. The second method is to reduce virtual keyboard operation, identify keyboard controls based on keyword (such as XCUIElementTypeKeyboard) matching of the virtual keyboard controls in the control tree, aggregate the keyboard controls into one control for operation, reduce the probability of keyboard operation, retain the keyboard input function, and avoid the keyboard from being covered for a long time by limiting the longest use time of the keyboard.

However, the above-described method has the following technical problems:

1. directly eliminating the virtual keyboard overlay would result in some of the controls and triggered events that would only appear if the virtual keyboard were invoked, not being tested. For example, controls that do not support input when the virtual keyboard is not tuned may support input after the virtual keyboard is tuned on. For example, when the virtual keyboard is not tuned up, controls such as "send", "publish" and the like associated with the input event cannot be displayed or cannot be operated, so that the controls and the event are missed.

2. Directly filtering out the virtual keyboard controls can cause the function class controls in the virtual keyboard that can trigger the event to be tested to also be filtered. For example, controls that can trigger events to be tested, such as "complete", "send", "confirm", etc., which are generally located at the lower right corner of a custom-like keyboard, are also filtered, resulting in that the virtual keyboard cannot be retracted and the events are missed. For example, the text content can be input by a wda input method, but the customized expression control still needs to be input by clicking a keyboard, and all filtering results in that the customized expression control cannot be tested.

3. The keyboard controls of the virtual keyboard are aggregated into one control, and the diversity of the keyboard control types is not considered. For example, the functional keyboard controls that can trigger events to be tested, such as "complete", "send", "confirm", etc., which are generally located at the lower right corner of the custom keyboard, have different test values from the common spelling keyboard controls, such as "a", "1", "space", etc., and a processing method for directly aggregating the controls into one control has a defect.

4. There is a possibility of eliminating the virtual keyboard hiding failure. The wda keyboard folding method or the downslide method has the problem that the virtual keyboard cannot be folded, so that the keyboard is covered for a long time, and if the keyboard scene is forcibly quitted only through cold start, the traversal efficiency is influenced to a certain degree. For example, in some custom virtual keyboard usage scenarios, a specific control must be clicked to exit the keyboard scenario.

5. There is a problem of a failure in the identification of the virtual keyboard control. The keyboard controls are various and comprise letters, expressions, characters displayed on the keyboard by automatically associating input contents, and the like, and the keyboard controls cannot be identified by a plurality of uniform and special keywords.

Based on the technical defects, the present disclosure provides a virtual keyboard processing method and apparatus, and further provides a plurality of keyboard processing modes. The plurality of processing modes process the virtual keyboard under different conditions, thereby improving the flexibility of processing the virtual keyboard. According to the continuous times of the virtual keyboard in the keyboard scene, the corresponding keyboard processing method is determined, the keyboard control depending on the virtual keyboard and the event related to the keyboard control can be tested, and meanwhile, the traversing efficiency is prevented from being influenced by the fact that the keyboard is covered for a long time.

The keyboard processing method provided by the application can be applied to the application environment shown in fig. 1. In which a terminal 101 communicates with a server 102 via a network. The terminal 101 may display a traversal test procedure of the application interface control function, and the server 102 may operate the test procedure and process the virtual keyboard. The terminal 101 may be, but not limited to, various personal computers, notebook computers, smart phones, tablet computers, and portable wearable devices, and the server 102 may be implemented by an independent server or a server cluster formed by a plurality of servers.

It should be noted that the terms "first," "second," and the like in the description and claims of the present disclosure and in the above-described drawings are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the disclosure described herein are capable of operation in sequences other than those illustrated or otherwise described herein. The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the disclosure, as detailed in the appended claims.

The following embodiments will explain the keyboard processing method and apparatus provided by the present disclosure with reference to the accompanying drawings.

Fig. 2 shows a flowchart of a virtual keyboard processing method provided in an embodiment of the present disclosure, and as shown in fig. 2, the method may include:

in step S210, during the traversal test of the application interface control function, the state of the virtual keyboard in the application interface control is detected.

In the embodiment of the disclosure, in the traversing test process of the application interface control function based on the terminal, a page crawling code source (pagesource) of a page is obtained. And analyzing the pagesource to obtain a control tree, wherein the control tree is a hierarchical structure consisting of controls. And determining the keyboard control keywords in the page by detecting the page pagesource, thereby carrying out scene detection on the virtual keyboard.

For example, the Keyboard control keyword may be a Keyboard. The virtual keyboard is detected through the keyboard control key words, and the accuracy of detecting the keyboard can be improved.

In the present disclosure, performing state detection on the virtual keyboard in the application interface control may be understood as performing scene detection on the virtual keyboard, that is, confirming whether the virtual keyboard is displayed in the current application interface.

It should be noted that the application scenario of the virtual keyboard in the present disclosure is an open state of the virtual keyboard.

In step S220, in response to detecting that the virtual keyboard is in the open state, a consecutive number of times that the virtual keyboard is continuously in the open state is determined.

In the embodiment of the present disclosure, the automated UI traversal test is composed of a series of consecutive test operations (such as clicking a control, sliding a page, etc.), and before each operation, whether the current page belongs to a keyboard scene (i.e., an open state) is detected. For example, the keyboard is detected by the 1 st operation, and the continuous times are 1;

detecting the keyboard by the 2 nd operation, wherein the continuous times are 2;

detecting the keyboard by the 3 rd operation, wherein the continuous times are 3;

the 4 th operation does not detect the keyboard, is not continuous any more, and the continuous times are 0;

the 5 th operation does not detect the keyboard, and the continuous times are 0;

detecting the keyboard by the 6 th operation, wherein the continuous times are 1;

detecting the keyboard by 7 th operation, wherein the continuous times are 2;

no keyboard is detected in the 8 th operation, the operation is not continuous, and the continuous times are 0.

And if the virtual keyboard is detected to be in the keyboard scene, recording the virtual keyboard in the keyboard scene, and determining the continuous times of the virtual keyboard in the keyboard scene.

In step S230, the virtual keyboard is processed according to the consecutive times.

In the embodiment of the disclosure, a processing mode of the virtual keyboard is determined according to the continuous times of the virtual keyboard in the keyboard scene, so as to process the virtual keyboard according to the determined processing mode.

According to the keyboard processing method, the method for processing the virtual keyboard can be determined by determining the continuous times of the virtual keyboard in the keyboard scene continuously, so that the keyboard control depending on the virtual keyboard and events related to the keyboard control can be tested, and the phenomenon that the traversing efficiency is influenced by the long-time covering of the keyboard is avoided.

In the embodiment of the present disclosure, a plurality of different thresholds for the number of continuous times of the virtual keyboard in the keyboard scene may be preset. And further comparing the continuous times of the virtual keyboard continuously in the keyboard scene with a preset threshold value to obtain the size relation between the continuous times and the threshold value. And determining a corresponding virtual keyboard processing mode according to the determined size relationship, and processing the virtual keyboard according to the determined virtual keyboard processing mode.

In the present disclosure, the following embodiments will explain an embodiment of processing a virtual keyboard based on a virtual keyboard processing manner corresponding to a size relationship.

Fig. 3 is a flowchart illustrating a keyboard processing method based on a click rate according to an embodiment of the present disclosure, and as shown in fig. 3, the method may include:

in step S310, in response to determining that the number of consecutive times is less than or equal to the first threshold, a keyboard control associated with the virtual keyboard is obtained, and the click rate of the keyboard control is adjusted based on the weight of the keyboard control.

In step S320, the virtual keyboard is processed by operating the keyboard control based on the click rate.

In the embodiment of the present disclosure, a plurality of thresholds are preset, and after the continuous number of times that the virtual keyboard is continuously in the on state (i.e., the keyboard scene) is determined, the continuous number of times may be first compared with the minimum value of the plurality of thresholds, and then compared with the thresholds one by one from small to large. Wherein the threshold is an integer value.

If the continuous times that the virtual keyboard is continuously positioned in the keyboard scene are determined to be smaller than or equal to a first threshold value, a keyboard control associated with the virtual keyboard is obtained, and the weight of the obtained keyboard control is further determined. And adjusting the click rate of the keyboard control based on the weight of the keyboard control. It should be noted that the first threshold value may be the minimum value of the preset threshold values in the present disclosure.

In the method, the keyboard controls are sorted according to the click rate of the keyboard controls, and the keyboard controls with high click rate are preferentially operated, so that the virtual keyboard is processed. Repeated and meaningless content input is avoided, and events depending on the keyboard control can be tested.

In the present disclosure, an embodiment of adjusting the click rate of a keyboard control is as follows.

Fig. 4 shows a flowchart of a method for adjusting a click rate of a keyboard control according to an embodiment of the present disclosure, and as shown in fig. 4, the method may include:

in step S410, the obtained keyboard controls are classified based on the preset keyboard control type.

In the embodiment of the present disclosure, the keyboard control type may be predetermined, for example, the keyboard control type may be a keyboard control of a functionality type, a spelling type, an expression type, and the like.

The functional keyboard controls can be sending, releasing, determining and the like; the spelling type keyboard controls may be letters, operators, numbers, etc., such as "A-Z", "spaces", "+", "1", etc.

In the present disclosure, a keyboard control type may be determined based on an event associated with the keyboard control.

And further classifying the obtained keyboard controls according to the preset keyboard control types.

In step S420, a weight preset for each keyboard control type is determined.

In the embodiment of the present disclosure, a corresponding weight may be determined for each keyboard control type, for example, a functionality type corresponds to a first weight, a spelling type corresponds to a second weight, an expression type corresponds to a third weight, and the like.

In step S430, the click rate of the classified keyboard controls is adjusted based on the weight.

In the embodiment of the present disclosure, after determining the weight corresponding to each keyboard control type, and determining the weight corresponding to the classified keyboard control, thereby adjusting the click rate of the classified keyboard control based on the weight of each keyboard control.

Illustratively, the weight of the spelling type keyboard control can be set to be zero, so that when the keyboard control is clicked, the spelling type keyboard control can be filtered, repeated and meaningless content input is avoided, and the risk that the test account is forbidden can be reduced.

Under the conditions that the number of the keyboard controls of the expression types is large and the value of repeated tests is not high, the expression controls can be aggregated into a control to share one weight.

A relatively high weight is set for the keyboard control of the functionality type to enable keyboard control dependent events to be tested.

In the embodiment of the present disclosure, the keyboard control corresponding to the maximum click rate may also be operated, and an event associated with the keyboard control is triggered. And executing the triggered event, and reserving the virtual keyboard.

Furthermore, a control with the highest weight is selected for operation through UI traversal based on the operation weight of the control, and the operation weight of the control is reduced along with the increase of the operation times. Sharing one operation weight can reduce the click probability of the expression keyboard control.

According to the method and the device, different measures are taken for different keyboard controls, so that triggering of more kinds of events and testing of more kinds of controls are achieved.

The following embodiments of the present disclosure will explain a manner of processing for canceling a keyboard.

In some embodiments of the present disclosure, in response to determining that the virtual keyboard is continuously in the keyboard scene for a number of consecutive times that is greater than a first threshold and less than or equal to a second threshold, the virtual keyboard is cancelled based on the gesture operation.

In other embodiments of the present disclosure, in response to determining that the continuous number of times that the virtual keyboard is continuously located in the keyboard scene is greater than the second threshold and less than or equal to the third threshold, the keyboard control of the virtual keyboard is acquired, and the keyboard control is operated to cancel the virtual keyboard.

In still other embodiments of the present disclosure, in response to determining that the number of consecutive times that the virtual keyboard is continuously in the keyboard scene is greater than a third threshold, the virtual keyboard is cancelled based on the cold start operation.

In embodiments of the present disclosure, the virtual keyboard may be cancelled based on the keyboard control.

Fig. 5 illustrates a flowchart of a method for canceling a virtual keyboard based on a keyboard control according to an embodiment of the present disclosure, where as shown in fig. 5, the method may include:

in step S510, a keyboard control of a function type is acquired, and the keyboard control of the function type is operated.

In step S520, the virtual keyboard is cancelled in response to triggering execution of the event associated with the keyboard control of the function type.

In the disclosed embodiments, the virtual keyboard may be cancelled through a function type keyboard control. For example, a user inputs content in a comment area of an application, and executes a sending action of the input content by triggering a sending control in a keyboard control, so that a virtual keyboard is triggered to be retracted, and a canceling effect of the virtual keyboard in a keyboard scene is realized.

The method for canceling the virtual keyboard can shorten the covering time of the keyboard, improve the success rate of retracting the keyboard and effectively control the covering time of the keyboard.

In the disclosed embodiment, the keyboard control can be obtained in different ways, and the implementation manner is as follows.

Fig. 6 shows a flowchart of a keyboard control acquiring method provided by an embodiment of the present disclosure, and as shown in fig. 6, the method may include:

in step S610, a keyboard control corresponding to the keyboard control keyword is obtained based on a preset keyboard control keyword.

In step S620, the coordinate range of the keyboard control is determined, and other keyboard controls within the coordinate range are obtained.

In the embodiment of the present disclosure, a keyboard control corresponding to a keyboard control keyword may be obtained in an analyzed control tree based on a preset keyboard control keyword.

And further determining the coordinate range of the keyboard control according to the determined keyboard control, and acquiring other keyboard controls in the coordinate range.

Illustratively, the keyboard control keyword may be a default naming convention, whether the xpath contains the XCUIElementTypeKeyboard control type, and the like. The coordinate range of the keyboard control may be identified based on the coordinate range of the XCUIElementTypeKeyboard class control. For example, the text appearing by the automatic association of the keyboard input content within 50 pixels above the keyboard can be regarded as the keyboard control based on the coordinate range of the XCUIElementTypeKeyboard control. For example, when the user inputs the text "i" on the keyboard, words related to "i" such as "i", etc. appear, and this part is that "i" is automatically associated with the keyboard control for input, that is, words such as "i", etc. also belong to the keyboard control, and then different operation weights are configured based on the keyboard control category to which the control belongs.

According to the method and the device, the keyboard coordinate range and the keyboard control keyword are combined, and the accuracy of identifying the keyboard control is improved.

In the embodiment of the present disclosure, after the virtual keyboard is processed, the scene detection is performed on the virtual keyboard again, and it is determined whether the virtual keyboard is cancelled successfully or whether the virtual keyboard is continuously located in the keyboard scene for a continuous number of times.

Fig. 7 is a general schematic diagram illustrating a method for canceling a virtual keyboard according to an embodiment of the present disclosure, as shown in fig. 7, including the following steps:

step 1, obtaining pagesource of the current page based on wda.

And 2, judging whether the Keyboard scene is in the Keyboard scene or not based on whether the pagesource contains the keyword Keyboard or not. If the keyboard scene exists, the step 3 is entered, and the continuous times of covering the keyboard are recorded. Otherwise, step 13 is entered, no keyboard-related processing is performed, and normal traversal is executed.

And 3, recording the continuous times of the continuous keyboard scene, and then adopting different keyboard processing methods based on the continuous times.

And 4, judging whether the continuous times of the continuous keyboard scenes exceed a times (the numerical value of a can be adjusted according to the actual traversal effect so as to trigger more events and test more controls to select the value of a as the target). If the number of times exceeds a, step 5 is entered, namely the keyboard is processed. Otherwise, step 11 is entered, i.e. the keyboard is not processed, and the keyboard is directly traversed in the keyboard scene.

And 5, judging whether the continuous times of the continuous keyboard scenes exceed a +1 times. And if the number of the steps exceeds a +1, entering the step 6 and carrying out the next judgment. Otherwise, go to step 9 and try the simpler keyboard elimination method.

Of course, the threshold may also be determined to be a +2 in the embodiment of the present disclosure, and is not specifically limited herein.

And 6, judging whether the continuous times of the continuous keyboard scenes exceed a +2 times. If the number of times exceeds a +2, the time for staying in the keyboard scene is too long, and the step 7 is entered to forcibly quit the keyboard scene. Otherwise, go to step 8 and try the more complicated keyboard elimination method.

And 7, cold starting, forcibly ending traversal in the keyboard scene, avoiding overlong stay time in the keyboard scene, and entering the step 10.

And 8, performing image recognition based on keyword matching control attributes or screenshot, identifying controls (such as "<", "," complete "," send "," release "," search "\ 8230;") which can be used for eliminating the keyboard (including closing the keyboard and triggering keyboard associated events \8230;, \8230;) in the page, operating the controls, and entering step 10.

And 9, eliminating the keyboard by adopting a wda method (such as sliding down and retracting the keyboard).

And step 10, acquiring the pagesource again, and judging whether the Keyboard is successfully eliminated or not based on the Keyboard keyword Keyboard.

And step 11, if the keyboard is detected, acquiring a coordinate range of the keyboard, and identifying the keyboard control, the control covered by the keyboard and the characters displayed by the keyboard input content in an automatic association manner, wherein the characters are \8230, 8230and the characters are automatically displayed on the basis of the keyboard control keyword XCUIElementTypeKeyBoard and the keyboard coordinate range.

And step 12, dividing the keyboard control types based on the event types associated with the keyboard control, wherein the keyboard control types comprise spelling types (such as 'A', 'space', '+', '1', automatically associated characters \8230; \8230), expression types, function types (such as 'sending', 'publishing', 'determining' \8230;), keyboard covering types and the like, and the operation weights are used for configuring different operation weights.

And step 13, configuring different operation weights based on the control attributes, types, operation times and the like.

And 14, acquiring the control with the highest weight, and operating.

And step 15, recording the operation times of the control.

According to the keyboard processing method, the variety of the keyboard control types is considered, the keyboard control types are divided according to the event types related to the controls, different processing methods are adopted for the keyboard controls of different types, and different operation weights are configured according to the test values. And the keyboard processing is suitable for a system keyboard and various self-defined keyboards, has popularization and generality, and can greatly improve the success rate of eliminating the keyboard and identifying the keyboard control.

Based on the same principle as the method shown in fig. 2, fig. 8 shows a schematic structural diagram of a virtual keyboard processing apparatus provided by an embodiment of the present disclosure, and as shown in fig. 8, the virtual keyboard processing apparatus 800 may include:

the detection module 801 is configured to perform state detection on a virtual keyboard in an application interface control in a traversal test process of the application interface control function; a determining module 802, configured to determine, in response to detecting that the virtual keyboard is in an open state, a consecutive number of times that the virtual keyboard is continuously in the open state; a processing module 803, configured to process the virtual keyboard according to the consecutive times.

In this embodiment of the present disclosure, the processing module 803 is configured to, in response to determining that the consecutive times are less than or equal to a first threshold, obtain a keyboard control associated with the virtual keyboard, and adjust a click rate of the keyboard control based on a weight of the keyboard control; and operating the keyboard control based on the click rate to process the virtual keyboard.

In this embodiment of the present disclosure, the processing module 803 is further configured to classify the acquired keyboard controls based on a preset keyboard control type; determining a preset weight for each keyboard control type; and adjusting the classified click rate of the keyboard control based on the weight.

In this embodiment of the present disclosure, the processing module 803 is configured to operate the keyboard control corresponding to the maximum click rate, and trigger an event associated with the keyboard control; and executing the event and reserving the virtual keyboard.

In this embodiment of the disclosure, the processing module 803 is configured to cancel the virtual keyboard based on a gesture operation in response to determining that the consecutive number of times is greater than a first threshold and less than or equal to a second threshold;

or

In response to the fact that the continuous times are larger than a second threshold and smaller than or equal to a third threshold, a keyboard control of the virtual keyboard is obtained, and the keyboard control is operated to cancel the virtual keyboard;

or

In response to determining that the number of consecutive times is greater than a third threshold, canceling the virtual keyboard based on a cold start operation.

In this embodiment of the present disclosure, the processing module 803 is configured to obtain a keyboard control of a function type, and operate the keyboard control of the function type; canceling the virtual keyboard in response to triggering execution of an event associated with a keyboard control of the function type.

In an embodiment of the present disclosure, the apparatus further includes: an obtaining module 804;

the obtaining module 804 is configured to obtain, based on a preset keyboard control keyword, a keyboard control corresponding to the keyboard control keyword; and determining the coordinate range of the keyboard control, and acquiring other keyboard controls in the coordinate range.

In this embodiment of the present disclosure, the detection module 801 is further configured to perform scene detection on the virtual keyboard again.

In the technical scheme of the disclosure, the acquisition, storage, application and the like of the personal information of the related user all accord with the regulations of related laws and regulations, and do not violate the good customs of the public order.

The present disclosure also provides an electronic device, a readable storage medium, and a computer program product according to embodiments of the present disclosure.

FIG. 9 illustrates a schematic block diagram of an example electronic device 900 that can be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the disclosure described and/or claimed herein.

As shown in fig. 9, the apparatus 900 includes a computing unit 901, which can perform various appropriate actions and processes in accordance with a computer program stored in a Read Only Memory (ROM) 902 or a computer program loaded from a storage unit 908 into a Random Access Memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the device 900 can also be stored. The calculation unit 901, ROM 902, and RAM 903 are connected to each other via a bus 904. An input/output (I/O) interface 905 is also connected to bus 904.

A number of components in the device 900 are connected to the I/O interface 905, including: an input unit 906 such as a keyboard, a mouse, and the like; an output unit 907 such as various types of displays, speakers, and the like; a storage unit 908 such as a magnetic disk, optical disk, or the like; and a communication unit 909 such as a network card, a modem, a wireless communication transceiver, and the like. The communication unit 909 allows the device 900 to exchange information/data with other devices through a computer network such as the internet and/or various telecommunication networks.

The computing unit 901 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various dedicated Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, and so forth. The computing unit 901 performs the respective methods and processes described above, such as the keyboard processing method. For example, in some embodiments, the keyboard processing method may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as storage unit 908. In some embodiments, part or all of a computer program may be loaded onto and/or installed onto device 900 via ROM 902 and/or communications unit 909. When the computer program is loaded into the RAM 903 and executed by the computing unit 901, one or more steps of the keyboard processing method described above may be performed. Alternatively, in other embodiments, the computing unit 901 may be configured to perform the keyboard processing method by any other suitable means (e.g., by means of firmware).

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.

Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program code may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program code, when executed by the processor or controller, causes the functions/acts specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user may provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), and the Internet.

The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server may be a cloud server, a server of a distributed system, or a server with a combined blockchain.

It should be understood that various forms of the flows shown above, reordering, adding or deleting steps, may be used. For example, the steps described in the present disclosure may be executed in parallel, sequentially or in different orders, and are not limited herein as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved.

The above detailed description should not be construed as limiting the scope of the disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made, depending on design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present disclosure should be included in the scope of protection of the present disclosure.

Claims

1. A virtual keyboard processing method, the method comprising:

in the traversal test process of the application interface control function, carrying out state detection on a virtual keyboard in the application interface control;

in response to detecting that the virtual keyboard is in an open state, determining a number of consecutive times that the virtual keyboard is continuously in an open state;

and processing the virtual keyboard according to the continuous times.

2. The method of claim 1, wherein said processing the virtual keyboard according to the consecutive number of times comprises:

in response to the fact that the continuous times are smaller than or equal to a first threshold value, a keyboard control associated with the virtual keyboard is obtained, and the click rate of the keyboard control is adjusted based on the weight of the keyboard control;

and operating the keyboard control based on the click rate to process the virtual keyboard.

3. The method of claim 2, wherein the adjusting the click rate of the keyboard control based on the weight of the keyboard control comprises:

classifying the obtained keyboard controls based on a preset keyboard control type;

determining a preset weight for each keyboard control type;

and adjusting the click rate of the classified keyboard controls based on the weight.

4. The method of claim 2, wherein the operating the keyboard control based on the click rate to process the virtual keyboard comprises:

operating the keyboard control corresponding to the maximum click rate, and triggering an event related to the keyboard control;

and executing the event and reserving the virtual keyboard.

5. The method of claim 1, wherein said processing the virtual keyboard according to the consecutive number of times comprises:

in response to determining that the number of consecutive times is greater than a first threshold and less than or equal to a second threshold, canceling the virtual keyboard based on a gesture operation;

or

6. The method of claim 5, wherein the obtaining a keyboard control of the virtual keyboard and operating the keyboard control to cancel the virtual keyboard comprises:

acquiring a keyboard control of a function type, and operating the keyboard control of the function type;

canceling the virtual keyboard in response to triggering execution of an event associated with a keyboard control of the function type.

7. The method of any of claims 2-6, wherein the method further comprises:

acquiring a keyboard control corresponding to the keyboard control keyword based on a preset keyboard control keyword;

and determining the coordinate range of the keyboard control, and acquiring other keyboard controls in the coordinate range.

8. The method of claim 2 or 5, wherein after processing the virtual keyboard according to the consecutive number, the method further comprises:

and detecting the scene of the virtual keyboard again.

9. A virtual keyboard processing apparatus, the apparatus comprising:

the detection module is used for carrying out state detection on the virtual keyboard in the application interface control in the traversal test process of the application interface control function;

the determining module is used for responding to the detection that the virtual keyboard is in the opening state, and determining the continuous times of the virtual keyboard in the opening state continuously;

and the processing module is used for processing the virtual keyboard according to the continuous times.

10. The apparatus of claim 9, wherein the processing module is configured to:

in response to determining that the number of consecutive times is less than or equal to a first threshold, acquiring a keyboard control associated with the virtual keyboard, and adjusting a click rate of the keyboard control based on a weight of the keyboard control;

11. The apparatus of claim 10, wherein the processing module is further configured to:

classifying the acquired keyboard controls based on preset keyboard control types;

determining a preset weight for each keyboard control type;

and adjusting the classified click rate of the keyboard control based on the weight.

12. The apparatus of claim 11, wherein the processing module is configured to:

and executing the event and reserving the virtual keyboard.

13. The apparatus of claim 9, wherein the processing module is configured to:

or

14. The apparatus of claim 13, wherein the processing module is configured to:

15. The apparatus of any of claims 10-14, wherein the apparatus further comprises: an acquisition module;

the acquisition module is used for acquiring a keyboard control corresponding to the keyboard control keyword based on a preset keyboard control keyword;

16. The apparatus of claim 10 or 13, wherein the detection module is further configured to:

and detecting the scene of the virtual keyboard again.

17. An electronic device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein, the first and the second end of the pipe are connected with each other,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-8.

18. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-8.

19. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1-8.