WO2020006676A1 - Blind input method using touch soft keyboard, computing apparatus, and storage medium - Google Patents

Blind input method using touch soft keyboard, computing apparatus, and storage medium Download PDF

Info

Publication number
WO2020006676A1
WO2020006676A1 PCT/CN2018/094203 CN2018094203W WO2020006676A1 WO 2020006676 A1 WO2020006676 A1 WO 2020006676A1 CN 2018094203 W CN2018094203 W CN 2018094203W WO 2020006676 A1 WO2020006676 A1 WO 2020006676A1
Authority
WO
WIPO (PCT)
Prior art keywords
input
character
touch
keyboard
input method
Prior art date
Application number
PCT/CN2018/094203
Other languages
French (fr)
Chinese (zh)
Inventor
喻纯
王丰
石伟男
范舒翼
王童
史元春
Original Assignee
清华大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 清华大学 filed Critical 清华大学
Priority to PCT/CN2018/094203 priority Critical patent/WO2020006676A1/en
Publication of WO2020006676A1 publication Critical patent/WO2020006676A1/en

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures

Definitions

  • the present invention generally relates to information input technology of a computing device, and particularly to a blind input technology using a soft keyboard on a touch screen device, a computing device, and a computer storage medium.
  • Text input on a touch screen device is one of the most basic functions of a smart device.
  • an input method using a touch-type soft keyboard in a computing device comprising: determining a coordinate position of a user's finger on a screen in response to a touch input signal from a touch sensor; and according to the touch position , Calculate the input probability of each character; determine the optimal character according to the input probability value of each character, and play the voice of the optimal character.
  • determining the optimal character includes: determining, according to the input probability value of each character, the character with the highest probability value as the candidate optimal character; and trying to update the keyboard layout according to the touch-down position and the determined candidate optimal character, So that the candidate optimal character is at the touch-press position and the updated keyboard layout meets a predetermined standard; if the updated keyboard layout is successful, the candidate optimal character is determined to be the optimal character; otherwise, the default standard keyboard layout is used to determine The optimal character is the key at the touched position in this layout. .
  • the input method further includes: in response to the user continuing to move to access other key characters without lifting the finger, playing the voice of the key character corresponding to the finger position.
  • the input method further includes: after playing the voice of the key character, in response to detecting a confirmation signal originating from the user, completing the input of the confirmed character.
  • the confirmation signal includes: a signal triggered by the user lifting a finger or a signal triggered by a double-click operation after lifting the finger.
  • determining the input probability of each character includes: calculating the input probability of each character based on the character that has been input and the position where the user touched and pressed, using a drop point model and a language model.
  • the method before calculating the input probability of each character according to the touch position, the method further includes: determining whether the distance between the current touch position and the previous confirmed input touch position is less than a certain threshold, and if so, directly using the previous Confirm the keyboard layout at the time of input, and do not update the keyboard layout; if not, restore the keyboard layout to the default standard keyboard layout, and then do probability calculations and determine the optimal characters, and update the keyboard layout as needed based on the results.
  • the keyboard layout is an invisible keyboard response area setting capable of achieving the same effect.
  • the landing point model is a normal distribution.
  • the updating the keyboard layout includes moving and / or compressing the keyboard.
  • the moving and / or compressing the keyboard includes: performing branch or overall movement and compression; compressing the keyboard in a linear or exponential manner with the touch-pressed position as the center; the movement must not exceed a predefined effective range; the compression must not Make any key size smaller than the size threshold.
  • the input method further includes determining whether a gesture input has occurred, the determining whether the gesture input has occurred includes: detecting a contact trajectory within a full screen range instead of being limited to an area occupied by a soft keyboard; using a displacement of the contact trajectory- The time relationship determines whether a gesture input has occurred and determines the type of gesture; based on the determined gesture type, a corresponding input operation is determined.
  • gesture types and corresponding input operations include: single-finger swipe left: delete the last typed character; two-finger swipe left: clear all typed characters; single-finger swipe up: switch the highlighted candidate to the previous candidate Words: Single-finger swipe: Switch the highlighted candidate to the next candidate; Single-finger swipe right: Confirm input of the current candidate
  • the input method further includes: providing audio feedback for touch operations, including: feedback on key presses; feedback on confirming input; and response to gesture operations.
  • a computing device including: a touch screen with a touch sensor capable of detecting a touch input; a controller and a memory, wherein the computer-executable instructions are stored on the memory, and the touch-type soft keyboard is activated
  • the computer-executable instructions are executed by a controller, they are operable to perform the aforementioned input method.
  • a computer-readable storage medium having computer-executable instructions stored thereon, and in a case where a touch-type soft keyboard of a computing device has been activated: When executed, it is operable to perform the aforementioned input method.
  • the feedback of the input method is completely audioized, and the prediction algorithm is adjusted.
  • the soft keyboard zoom and translation can be implemented according to the characteristics of blind input, which can effectively improve Input accuracy and speed of blind people on touch screen devices.
  • the techniques used in this example are based on conditional probability formulas, Bayesian formulas, text-to-speech synthesis, and so on.
  • FIG. 1 illustrates an overall flowchart of an input method 100 for inputting by using a touch-type soft keyboard in a computing device according to an embodiment of the present invention.
  • FIG. 4 shows an overall flow of a text input method 200 according to another embodiment of the present invention.
  • Figures 5a and 5b show examples of soft keyboard movement and zooming processes in actual use.
  • FIG. 6 shows a schematic example of illegal movement of a soft keyboard in actual use.
  • Fig. 7a shows the keyboard state before the user presses a key in an input example
  • Fig. 7b shows the layout of the keyboard after the user first presses a key when entering a letter
  • Fig. 7c shows an error that occurs when "o" is entered situation.
  • Figure 7d uses the left swipe gesture to delete, at this time the voice feedback: "o has been deleted”.
  • Touch-press refers to the action from the finger never touching the touch screen to touching the touch screen.
  • Movement Refers to the movement of the finger while changing the position while keeping touching the touch screen.
  • Lift refers to the movement of a finger from touching the touch screen to not touching the touch screen.
  • FIG. 1 illustrates an overall flowchart of an input method 100 for inputting by using a touch-type soft keyboard in a computing device according to an embodiment of the present invention.
  • the input method of this embodiment is particularly suitable for inputting text.
  • the type of input text There is no limitation on the type of input text.
  • the input method may be English character input or Chinese character input.
  • step S110 in response to a touch input signal from a touch sensor, a coordinate position of a user's finger on a screen is determined.
  • step S120 the input probability of each character is calculated based on the touched position.
  • the input probability of each character currently input by the user is calculated based on the character string that has been input and the position where the user touches and presses, and uses a drop point model and a language model.
  • the language model is a Chinese / English dictionary consisting of vocabulary and frequency pairs.
  • the table shown in Figure 2 gives a schematic example of a language model.
  • the landing point model is a combined list of expected input keys and actual input key distributions on a touch-screen device when the user has no visual feedback. This model can be obtained through testing and statistics of user input behavior.
  • the table shown in Figure 3 gives a schematic example of the fall point model.
  • the N + 1th input result is combined to predict the N + 1th user input intention.
  • P (KI) is the probability that the user intends to press the button I.
  • Is the drop point model factor that is, the probability that the user enters key A when he wants to enter key I.
  • Is the language model factor that is, in the dictionary, all words with a length greater than or equal to N + 1 and the first N characters are the same characters that the user has entered, and the N + 1th character is the probability of pressing the key I.
  • the keys After calculating the probability values of all keys, the keys are sorted. The key with the highest probability is the key that the user expects to type, which is the optimal character described below.
  • step S120 the optimal character is determined according to the input probability value of each character, and the voice of the optimal character is played.
  • determining the optimal character in addition to considering the input probability value of each character, the touch-down position is also considered.
  • determining the optimal character includes: determining a character with the highest probability value as a candidate optimal character according to the input probability value of each character; and attempting to update the keyboard layout according to the touch-down position and the determined candidate optimal character to Make the candidate optimal character at the touch-press position and the updated keyboard layout meet predetermined criteria; if the updated keyboard layout is successful, determine the candidate optimal character as the optimal character; otherwise, use the default standard keyboard layout to determine the most optimal character.
  • the top character is the key at the touched position in this layout.
  • the input of the confirmed character is completed in response to detecting a confirmation signal originating from the user.
  • the confirmation signal is a signal triggered by a user gesture, such as a signal triggered by a user lifting a finger or a signal triggered by a double-click operation after lifting a finger.
  • a user gesture such as a signal triggered by a user lifting a finger or a signal triggered by a double-click operation after lifting a finger.
  • the confirmation signal is not limited to a signal triggered by a gesture.
  • the gesture is not limited to a gesture such as raising a finger or clicking after the finger is raised.
  • the text input method improves the accuracy of Chinese and English input without visual feedback, and uses voice to interact with the user, which is very suitable for blind people to use.
  • the text input method can be applied, for example, to a computing device such as a smart phone or a tablet computer.
  • FIG. 4 shows an overall flow of a text input method 200 according to another embodiment of the present invention.
  • step S210 the coordinate position of the user's finger on the screen is determined in response to a touch input signal from the touch sensor.
  • step S220 it is determined whether the distance between the current touch position and the previous touch lift position is less than a certain threshold.
  • the close position indicates that the user will next enter the same letter or the character next to the last key input.
  • the keyboard layout should be updated. For example, if the two input positions are basically the same, the current touch is considered to have occurred on the keyboard layout when it was last lifted, and the keys to be input are the same as the last time. For example, in "hello”, the last time you entered “L”, and the current touch position is still "L”, the two inputs are considered to be related, and there is no need to update the keyboard layout. After the update, it may lead to the inability to enter L correctly; another This situation is basically a key difference between the two input positions.
  • the blind user wants to input the adjacent letter of the last entered letter.
  • the blind user wants to enter "bed” and the last time “e” is input.
  • the position is the position below the last touched position, it is considered that the blind person wants to enter "d” this time.
  • step S220 determines whether the movement range does not exceed a certain threshold. If it exceeds a certain threshold, the process proceeds to step S230.
  • step S230 an input character of each character is calculated based on the touched position.
  • step S230 reference may be made to the description made in conjunction with step S120 in FIG.
  • step S250 the candidate optimal character is determined according to the input probability value of each character.
  • step S260 an attempt is made to update the keyboard layout as appropriate so that the candidate optimal character is at the touch position, and it is determined whether the keyboard layout meets a predetermined standard.
  • Updated keyboard layouts here include, for example, mobile and / or compressed keyboards.
  • the interaction between the soft keyboard and the user is adjusted by moving and zooming the soft keyboard, keeping the user's memory of the relative position of the keyboard during the key adjustment process, and improving the efficiency of the user's input adjustment.
  • Moving and / or compressing the keyboard may include: performing branch or overall movement and compression; compressing the keyboard in a linear or exponential manner with the touch position as the center, and the linear or exponential manner is user selectable.
  • the system After the system makes a request to update or adjust the soft keyboard, it will determine whether the request is legal, in other words, whether it meets the predetermined criteria.
  • the criterion for this judgment can be: the movement must not exceed the predefined effective range; the compression must not make any key size smaller than the size threshold that the user can touch, such as whether any key size is smaller than a certain percentage of the original size. If such a key exists later, the request is illegal and no adjustment will be made. If the request is valid, the keyboard will actually be adjusted.
  • the adjustment method is that the entire keyboard is scaled and translated linearly, so that the center of the target key is adjusted to the target position.
  • the movement must not exceed a predefined effective range, which can be determined by using, for example, an input range of a current commonly used input method as a reference.
  • step S260 if it is determined in step S260 that the keyboard layout meets the predetermined standard, proceed to step S280, that is, use the updated keyboard layout, confirm that the candidate optimal character is the optimal character, and play the optimal character, otherwise proceed to step In S270, a default keyboard layout is adopted, and the key corresponding to the touch position under the default keyboard layout is determined to be the optimal character, and the optimal character voice is played.
  • Figures 5a and 5b show examples of soft keyboard movement and zooming processes in actual use. This is the scene of entering the fourth pinyin letter when typing the word "PENGYOU”.
  • the actual touch key is "F”
  • the algorithm predicts that the user intends to type "G”
  • the algorithm requests the keyboard to move the center of the G key to the current touch point.
  • the keyboard confirms that this movement is legal, and finally realizes the movement of the soft keyboard.
  • the resulting voice feedback is "G”.
  • the dark gray circle on the letter G in FIG. 5b indicates the finger touch position.
  • the dark gray circles in Figs. 7b, 7c also indicate the finger touch position.
  • FIG. 6 shows a schematic example of illegal movement of a soft keyboard in actual use. Specifically, when the last letter s of the company is input, since the key position is too left, compressing the key will cause the a key width to be smaller than the minimum value (for example, 50% of the original key width), so the movement is illegal. Although the probability maximum key bit is s, because the movement is illegal, the final input key is the key a corresponding to the pressed position.
  • the minimum value for example, 50% of the original key width
  • step S290 the event is continuously monitored. If a mobile event is received, voice feedback is given during the movement. For example, after the system gives a voice prompt for the optimal character, the user thinks that this is not the character he wants, at this time the finger will continue to move to find the remaining characters, and in the process, the voice feedback will continue to be given.
  • step S291 in response to receiving the user lifting event, a gesture judgment is performed, and a voice prompt is given.
  • operations such as deleting, confirming word selection, switching candidate words, and clearing are implemented through gesture operations.
  • Determining whether a gesture input has occurred includes detecting contact trajectories in a full screen range instead of being limited to the area occupied by the soft keyboard; using the displacement-time relationship of the contact trajectories to determine whether gesture input has occurred and determining the type of gesture, so that the Type of gesture while determining the corresponding input operation.
  • the gesture type may include: single-finger swipe left, two-finger swipe left, single-finger swipe up, single-finger swipe down, single-finger swipe right, and so on.
  • gestures and operations can be as follows:
  • Single-finger left stroke In the Chinese multi-step word selection, the previous word selection is canceled. In other cases, the last pinyin or letter of the currently selected word is not confirmed. If not, the last Chinese character or letter that completed the word selection is deleted.
  • One-finger swipe up Set the currently highlighted candidate as the previous candidate in the list.
  • One-finger swipe down Set the currently highlighted candidate as the next candidate in the list.
  • corresponding voice feedback is given, for example, when it is determined that the two-finger left-swipe gesture is performed, the voice prompt "clear all current inputs".
  • step S292 the operation corresponding to the determined gesture action is performed, and the information related to the lifting action is also recorded under the clearing determined as the input operation.
  • the character is input, and the related information of the current lifting is recorded for comparison of the next input.
  • step S210 When a subsequent touch event is received from the user, the process starts from step S210 again.
  • the user input is predicted by combining the language model and the drop point model, and the input without visual feedback is adapted by moving and zooming the soft keyboard, using voice to interact with the user, and using gesture operations to complete the text Word selection, confirmation, deletion, and clearing operations during input can improve the accuracy of Chinese and English input without visual feedback and the speed of Chinese and English input without visual feedback.
  • the keyboard layout is not limited to a visually visible keyboard layout, and may also be an invisible keyboard response area setting capable of achieving the same effect, which has the same effect for blind people.
  • Figure 7a shows the keyboard state before the user presses a key.
  • the language model and the drop point model will be used to calculate the key with the highest probability that the user expects to press, and the center of the key will be moved to the user touch point by zooming and panning the keyboard.
  • the voice feedback content is the current key, such as 'H'. (The prediction order is displayed on the keyboard through the corner mark, for debugging and display use, non-feedback content)
  • Figure 7b shows the layout of the keyboard after the user first presses a letter when entering a letter.
  • the key can be retrieved by moving the finger while keeping the finger pressed.
  • the keys passed by the user's finger will be feedback in real time through voice, and the ongoing voice feedback will be interrupted by the new voice.
  • the keyboard layout is maintained.
  • Fig. 7c shows a situation where an error occurs when "o" is input.
  • "i” is input by mistake and deleted with a left swipe gesture.
  • the reference numeral 1 drawn by the horizontal line indicates the content currently input according to the embodiment of the present invention, that is, the content confirmed by the user.
  • the input is confirmed by the user's lifting action
  • the reference numeral 2 indicates according to the embodiment of the present invention.
  • reference numeral 3 indicates the input content without running the embodiment of the present invention, that is, the content input by touching the touch point according to the default layout.
  • the number 4 indicates that the currently entered character is i
  • the number 5 indicates candidate words, including heli, hellish, helling, and hellip.
  • the numbers displayed on the characters of the soft keyboard shown below indicate the input probability calculated by the probability ranking. Five characters, that is, the character with the first probability of input is I, the character with the second probability of input is O, the character with the third probability is U, the character with the fourth probability is J, and the character with the fifth probability Is Y.
  • the user can use the up and down gestures to select the previous and next candidate words, and at the same time, the voice feedback changes the candidate words.
  • This function is less useful in English input, mainly used when there are homophones in Chinese input.
  • the user can use a two-finger left swipe gesture to clear all input.
  • the voice feedback "empties".
  • FIG. 7d schematically illustrates the deletion of “o” using the left swipe gesture.
  • the voice feedback “o has been deleted”, the circle in FIG. 7d And the left arrow gestures to the left.
  • the input method of this embodiment is particularly suitable for use by blind people, but of course it can also be used by people with normal vision.
  • gestures and corresponding operations are merely examples, and other gestures may be designed and corresponding to different operations as required.
  • the input method of the embodiment of the present invention can be applied to various computing devices, such as a smart phone, a tablet computer, and the like.
  • Embodiments of the present invention may also be implemented as a computing device, including: a touch screen with a touch sensor capable of detecting a touch input; a controller and a memory, the computer-executable instructions stored on the memory, and when the touch soft keyboard is activated:
  • the computer-executable instructions, when executed by a controller, are operable to perform the input method described above.
  • Embodiments of the present invention may also be implemented as a computer-readable storage medium having computer-executable instructions stored thereon.
  • the touch-type soft keyboard of a computing device has been activated: when the computer-executable instructions are executed by a computer, Operable to perform the input method described above.

Abstract

Provided are an input method for implementing input using a touch soft keyboard, a computing apparatus, and a computer readable medium. The input method comprises determining the coordinate position of a finger of a user on a screen in response to a touch input signal from a touch sensor; on the basis of the touch position, calculating the input probability of each character; on the basis of the input probability value of each character, determining the optimum character, and playing a sound of the optimum character. Furthermore, determining the optimum character comprises: on the basis of the input probability value of each character, determining the character having the highest probability value to be a candidate optimum character; on the basis of the touch press position and the determined candidate optimum character, attempting to update the keyboard layout, such that the candidate optimum character is in the touch press position and the updated keyboard layout meet a predetermined standard; if the keyboard layout update is successful, then determining that the candidate optimum character is the optimum character; and, if not, then using a default standard keyboard layout, and determining that the optimum character is the button in the touch press position in said layout.

Description

利用触摸软键盘的盲式输入法、计算装置和存储介质Blind input method using touch soft keyboard, computing device and storage medium 技术领域Technical field
本发明总体地涉及计算设备的信息输入技术,特别是涉及触屏设备上利用软键盘的盲式输入技术、计算装置和计算机存储介质。The present invention generally relates to information input technology of a computing device, and particularly to a blind input technology using a soft keyboard on a touch screen device, a computing device, and a computer storage medium.
背景技术Background technique
在触屏设备上进行文字输入是智能设备最基本的功能之一。Text input on a touch screen device is one of the most basic functions of a smart device.
存在使用语言模型实现对输入的预测的现有技术,如CN102915122A。该现有技术默认键入操作的正确性,通过视觉进行反馈,不适用盲人输入应用。There are existing technologies, such as CN102915122A, that use language models to achieve predictions of inputs. This prior art defaults the correctness of the typing operation and feedbacks through vision, which is not suitable for blind input applications.
也存在使用直接屏幕阅读的方式使得盲人可以在触屏设备上进行输入的技术,例如专利CN103064529A,该现有技术选词操作同键入操作同为手指在屏幕上检索,增加了误操作的可能性,盲人输入的实用性较低;以及无法对盲人意图进行预测,无法进行纠错,降低输入准确性,增加了检索按键的耗时。There are also technologies that use direct screen reading to enable blind people to input on touch-screen devices, such as the patent CN103064529A. This prior art word selection operation is the same as typing operation on the screen to search with fingers, increasing the possibility of misoperation. , The practicability of blind input is low; and the intent of the blind cannot be predicted, error correction cannot be performed, the accuracy of the input is reduced, and the time-consuming to retrieve the keys is increased.
发明内容Summary of the invention
鉴于上述情况,提出了本发明。In view of the foregoing, the present invention has been made.
根据本发明的一个方面,提供了一种计算装置中利用触摸式软键盘进行输入的输入方法,包括:响应于来自触摸传感器的触摸输入信号,确定用户手指在屏幕上的坐标位置;根据触摸位置,计算各字符的输入概率;根据各字符的输入概率值,确定最优字符,并播放该最优字符的语音。According to an aspect of the present invention, there is provided an input method using a touch-type soft keyboard in a computing device, comprising: determining a coordinate position of a user's finger on a screen in response to a touch input signal from a touch sensor; and according to the touch position , Calculate the input probability of each character; determine the optimal character according to the input probability value of each character, and play the voice of the optimal character.
可选地,所述确定最优字符包括:根据各字符的输入概率值,确定概率值最大的字符作为候选最优字符;根据触摸按下位置和确定的候选最优字符,尝试更新键盘布局,以使得所述候选最优字符在所述触摸按下位置且更 新后的键盘布局满足预定标准;如果更新键盘布局成功,则确定候选最优字符为最优字符;否则采用默认标准键盘布局,确定最优字符为该布局下的触摸按下位置的按键。。Optionally, determining the optimal character includes: determining, according to the input probability value of each character, the character with the highest probability value as the candidate optimal character; and trying to update the keyboard layout according to the touch-down position and the determined candidate optimal character, So that the candidate optimal character is at the touch-press position and the updated keyboard layout meets a predetermined standard; if the updated keyboard layout is successful, the candidate optimal character is determined to be the optimal character; otherwise, the default standard keyboard layout is used to determine The optimal character is the key at the touched position in this layout. .
可选地,输入方法还包括:响应于用户不抬起手指继续移动访问其他按键字符,播放手指位置所对应按键字符的语音。Optionally, the input method further includes: in response to the user continuing to move to access other key characters without lifting the finger, playing the voice of the key character corresponding to the finger position.
可选地,输入方法还包括:在播放按键字符的语音之后,响应于检测到源于用户的确认信号,而完成被确认的字符的输入。Optionally, the input method further includes: after playing the voice of the key character, in response to detecting a confirmation signal originating from the user, completing the input of the confirmed character.
可选地,确认信号包括:用户抬起手指触发的信号或者抬起手指后双击点击操作触发的信号。Optionally, the confirmation signal includes: a signal triggered by the user lifting a finger or a signal triggered by a double-click operation after lifting the finger.
可选地,确定各字符的输入概率包括:基于已经输入的字符和用户触摸按下位置,结合使用落点模型与语言模型,计算各字符的输入概率。Optionally, determining the input probability of each character includes: calculating the input probability of each character based on the character that has been input and the position where the user touched and pressed, using a drop point model and a language model.
可选地,在所述根据触摸位置计算各字符的输入概率之前,还包括:确定本次触摸位置与前次确认输入的触摸位置之间的距离是否小于一定阈值,如果是,则直接使用前次确认输入时的键盘布局,不对键盘布局做更新;如果否,则将键盘布局恢复为默认标准键盘布局,然后再做概率计算和确定最优字符,并根据该结果视需要更新键盘布局。Optionally, before calculating the input probability of each character according to the touch position, the method further includes: determining whether the distance between the current touch position and the previous confirmed input touch position is less than a certain threshold, and if so, directly using the previous Confirm the keyboard layout at the time of input, and do not update the keyboard layout; if not, restore the keyboard layout to the default standard keyboard layout, and then do probability calculations and determine the optimal characters, and update the keyboard layout as needed based on the results.
可选地,所述键盘布局为不可见的能够达到相同效果的键盘响应区域设置。Optionally, the keyboard layout is an invisible keyboard response area setting capable of achieving the same effect.
可选地,所述落点模型为正态分布。Optionally, the landing point model is a normal distribution.
可选地,所述更新键盘布局包括移动和/或压缩键盘。Optionally, the updating the keyboard layout includes moving and / or compressing the keyboard.
可选地,所述移动和/或压缩键盘包括:进行分行或整体移动、压缩;以触摸按下位置为中心按照线性或指数方式对键盘进行压缩;移动不得超过预定义的有效范围;压缩不得使任何按键尺寸小于尺寸阈值。Optionally, the moving and / or compressing the keyboard includes: performing branch or overall movement and compression; compressing the keyboard in a linear or exponential manner with the touch-pressed position as the center; the movement must not exceed a predefined effective range; the compression must not Make any key size smaller than the size threshold.
可选地,输入方法还包括确定是否发生了手势输入,所述确定是否发生了手势输入包括:在全屏范围内而非局限于软键盘所占据区域检测触点轨迹;利用触点轨迹的位移-时间关系判断是否发生手势输入并判断手势类型;基于判定的手势类型,而确定对应的输入操作。Optionally, the input method further includes determining whether a gesture input has occurred, the determining whether the gesture input has occurred includes: detecting a contact trajectory within a full screen range instead of being limited to an area occupied by a soft keyboard; using a displacement of the contact trajectory- The time relationship determines whether a gesture input has occurred and determines the type of gesture; based on the determined gesture type, a corresponding input operation is determined.
可选地,手势类型以及对应的输入操作包括:单指左划:删除最后键入的字符;双指左划:清空所有键入的字符;单指上划:将高亮的候选词切换至上一个候选词;单指下划:将高亮的候选词切换至下一个候选词;单指右 划:确认输入当前候选词Optionally, gesture types and corresponding input operations include: single-finger swipe left: delete the last typed character; two-finger swipe left: clear all typed characters; single-finger swipe up: switch the highlighted candidate to the previous candidate Words: Single-finger swipe: Switch the highlighted candidate to the next candidate; Single-finger swipe right: Confirm input of the current candidate
可选地,输入方法还包括:对于触摸操作提供音频反馈,包括:对按键按下的反馈;对确认输入的反馈;对手势操作的反聩。Optionally, the input method further includes: providing audio feedback for touch operations, including: feedback on key presses; feedback on confirming input; and response to gesture operations.
根据本发明的另一方面,还提供了一种计算装置,包括:触摸屏,具有触摸传感器,能够检测触摸输入;控制器和存储器,存储器上存储有计算机可执行指令,在触摸式软键盘已启动的情况下:当所述计算机可执行指令当被控制器执行时,可操作来执行前述输入方法。According to another aspect of the present invention, there is also provided a computing device including: a touch screen with a touch sensor capable of detecting a touch input; a controller and a memory, wherein the computer-executable instructions are stored on the memory, and the touch-type soft keyboard is activated In the case of: when the computer-executable instructions are executed by a controller, they are operable to perform the aforementioned input method.
根据本发明另一方面,提供了一种计算机可读存储介质,其上存储有计算机可执行指令,在计算装置的触摸式软键盘已启动的情况下:当所述计算机可执行指令当被计算机执行时,可操作来执行前述输入方法。According to another aspect of the present invention, there is provided a computer-readable storage medium having computer-executable instructions stored thereon, and in a case where a touch-type soft keyboard of a computing device has been activated: When executed, it is operable to perform the aforementioned input method.
根据本发明实施例的输入法、计算装置和计算机可读介质,将输入法的反馈完全音频化,并对预测算法做出调整,针对盲人输入的特点实现了软键盘的缩放平移,可以有效提升盲人在触屏设备上的输入准确率和速度。本例中使用的技术基于条件概率公式、贝叶斯公式、文字转语音合成等。According to the input method, the computing device and the computer-readable medium of the embodiment of the present invention, the feedback of the input method is completely audioized, and the prediction algorithm is adjusted. The soft keyboard zoom and translation can be implemented according to the characteristics of blind input, which can effectively improve Input accuracy and speed of blind people on touch screen devices. The techniques used in this example are based on conditional probability formulas, Bayesian formulas, text-to-speech synthesis, and so on.
附图说明BRIEF DESCRIPTION OF THE DRAWINGS
从下面结合附图对本发明实施例的详细描述中,本发明的这些和/或其它方面和优点将变得更加清楚并更容易理解,其中:These and / or other aspects and advantages of the present invention will become clearer and easier to understand from the following detailed description of the embodiments of the present invention in conjunction with the accompanying drawings, in which:
图1示出了根据本发明实施例的计算装置中利用触摸式软键盘进行输入的输入方法100的总体流程图。FIG. 1 illustrates an overall flowchart of an input method 100 for inputting by using a touch-type soft keyboard in a computing device according to an embodiment of the present invention.
图2所示的表格给出了语言模型的一个示意性样例。The table shown in Figure 2 gives a schematic example of a language model.
图3所示的表格给出了落点模型的一个示意性样例。The table shown in Figure 3 gives a schematic example of the fall point model.
图4示出了根据本发明另一实施例的文本输入方法200的总体流程。FIG. 4 shows an overall flow of a text input method 200 according to another embodiment of the present invention.
图5a和图5b展示了实际使用中,软键盘移动、缩放的过程示例。Figures 5a and 5b show examples of soft keyboard movement and zooming processes in actual use.
图6示出了实际使用中,软键盘移动非法的一个示意性示例。FIG. 6 shows a schematic example of illegal movement of a soft keyboard in actual use.
图7a示出了一个输入示例中用户按下按键前键盘状态,图7b示出了用户在输入一个字母时,首次按下后键盘的布局,图7c示出了输入“o”时发生错误的情形。图7d使用左划手势删除,此时语音反馈:“o已删除”。Fig. 7a shows the keyboard state before the user presses a key in an input example, Fig. 7b shows the layout of the keyboard after the user first presses a key when entering a letter, and Fig. 7c shows an error that occurs when "o" is entered situation. Figure 7d uses the left swipe gesture to delete, at this time the voice feedback: "o has been deleted".
具体实施方式detailed description
为了使本领域技术人员更好地理解本发明,下面结合附图和具体实施方式对本发明作进一步详细说明。In order to enable those skilled in the art to better understand the present invention, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.
在介绍之前,解释一下有关术语在本文中的含义。Before introducing, explain the meaning of related terms in this article.
触摸按下:指手指从未接触触摸屏到接触触摸屏的动作。Touch-press: refers to the action from the finger never touching the touch screen to touching the touch screen.
移动:指手指在保持触摸触摸屏的状态下变化位置的动作。Movement: Refers to the movement of the finger while changing the position while keeping touching the touch screen.
抬起:指手指从触摸触摸屏到不触摸触摸屏的动作。Lift: refers to the movement of a finger from touching the touch screen to not touching the touch screen.
下面将参考附图描述本发明的具体实施例。Specific embodiments of the present invention will be described below with reference to the drawings.
图1示出了根据本发明实施例的计算装置中利用触摸式软键盘进行输入的输入方法100的总体流程图。FIG. 1 illustrates an overall flowchart of an input method 100 for inputting by using a touch-type soft keyboard in a computing device according to an embodiment of the present invention.
本实施例的输入方法特别适用于输入文字,这里对于输入文字的类型没有限制,例如可以是英文字符输入或者中文汉字输入。The input method of this embodiment is particularly suitable for inputting text. There is no limitation on the type of input text. For example, the input method may be English character input or Chinese character input.
如图1所示,在步骤S110中,响应于来自触摸传感器的触摸输入信号,确定用户手指在屏幕上的坐标位置。As shown in FIG. 1, in step S110, in response to a touch input signal from a touch sensor, a coordinate position of a user's finger on a screen is determined.
在步骤S120中,根据触摸按下位置,计算各字符的输入概率。In step S120, the input probability of each character is calculated based on the touched position.
在一个示例中,基于已经输入的字符串和用户触摸按下位置,结合使用落点模型与语言模型,计算用户当前输入各字符的输入概率。In one example, the input probability of each character currently input by the user is calculated based on the character string that has been input and the position where the user touches and presses, and uses a drop point model and a language model.
在一个示例中,语言模型是由词汇、出现频率成对组成的中/英文词典。图2所示的表格给出了语言模型的一个示意性样例。In one example, the language model is a Chinese / English dictionary consisting of vocabulary and frequency pairs. The table shown in Figure 2 gives a schematic example of a language model.
在一个示例中,落点模型是用户在无视觉反馈状态下,在触屏设备上,期望输入按键与实际输入按键分布的组合列表。这一模型可以通过对用户输入行为的测试、统计得到。图3所示的表格给出了落点模型的一个示意性样例。In one example, the landing point model is a combined list of expected input keys and actual input key distributions on a touch-screen device when the user has no visual feedback. This model can be obtained through testing and statistics of user input behavior. The table shown in Figure 3 gives a schematic example of the fall point model.
在一个示例中,通过结合语言模型与落点模型计算,对用户在单个单词/词汇拼音输入过程中的输入行为进行纠错与预测,提高输入结果的准确性并减少输入操作、修正操作次数,提高输入速度。一个具体的计算方式示例如下:In one example, by combining the calculation of the language model and the point model to correct and predict the input behavior of the user in the input process of a single word / vocabulary, improve the accuracy of the input result and reduce the number of input operations and correction operations. Increase input speed. An example of a specific calculation method is as follows:
在用户键入一个单词/词汇拼音的第N+1个字符时,将结合前N次的输入结果对第N+1次用户输入意图进行预测。首先计算键盘上每一个按键可能为用户输入意图的概率,在用户第N+1次输入键入键A后,单个按键概率的计算方式如下:When the user types the N + 1th character of a word / vocabulary, the N + 1th input result is combined to predict the N + 1th user input intention. First calculate the probability that each key on the keyboard may be the user's input intention. After the user enters key A at the N + 1th time, the probability of a single key is calculated as follows:
Figure PCTCN2018094203-appb-000001
Figure PCTCN2018094203-appb-000001
其中,P(KI)为用户意图为按键I的概率。
Figure PCTCN2018094203-appb-000002
为落点模型因子,即用户在希望输入键I时键入键A的概率。
Figure PCTCN2018094203-appb-000003
为语言模型因子,即字典中,所有长度大于等于N+1且前N个字符为用户已输入字符相同的单词中,第N+1个字符为按键I的概率。
Among them, P (KI) is the probability that the user intends to press the button I.
Figure PCTCN2018094203-appb-000002
Is the drop point model factor, that is, the probability that the user enters key A when he wants to enter key I.
Figure PCTCN2018094203-appb-000003
Is the language model factor, that is, in the dictionary, all words with a length greater than or equal to N + 1 and the first N characters are the same characters that the user has entered, and the N + 1th character is the probability of pressing the key I.
计算得到所有按键的这一概率值后,对其进行排序,概率最高的按键即为预测用户期望键入的按键,即下文所说的最优字符。After calculating the probability values of all keys, the keys are sorted. The key with the highest probability is the key that the user expects to type, which is the optimal character described below.
在步骤S120中,根据各字符的输入概率值,确定最优字符,并播放该最优字符的语音。In step S120, the optimal character is determined according to the input probability value of each character, and the voice of the optimal character is played.
在一个示例中,确定最优字符过程中,除了考虑各字符的输入概率值,还考虑触摸按下位置。具体地,例如,确定最优字符包括:根据各字符的输入概率值,确定概率值最大的字符作为候选最优字符;根据触摸按下位置和确定的候选最优字符,尝试更新键盘布局,以使得所述候选最优字符在所述触摸按下位置且更新后的键盘布局满足预定标准;如果更新键盘布局成功,则确定候选最优字符为最优字符;否则采用默认标准键盘布局,确定最优字符为该布局下的触摸按下位置的按键。。In one example, in determining the optimal character, in addition to considering the input probability value of each character, the touch-down position is also considered. Specifically, for example, determining the optimal character includes: determining a character with the highest probability value as a candidate optimal character according to the input probability value of each character; and attempting to update the keyboard layout according to the touch-down position and the determined candidate optimal character to Make the candidate optimal character at the touch-press position and the updated keyboard layout meet predetermined criteria; if the updated keyboard layout is successful, determine the candidate optimal character as the optimal character; otherwise, use the default standard keyboard layout to determine the most optimal character. The top character is the key at the touched position in this layout. .
在一个示例中,在播放按键字符的语音之后,响应于检测到源于用户的确认信号,而完成被确认的字符的输入。In one example, after playing the voice of the keystroke character, the input of the confirmed character is completed in response to detecting a confirmation signal originating from the user.
在一个示例中,确认信号是由用户手势触发的信号,例如用户抬起手指触发的信号或者抬起手指后双击点击操作触发的信号。不过此仅为示例,确认信号不限于是手势触发的信号,在手势触发的情况下,手势也不限于抬起手指或者抬起手指后点击操作这样的手势。In one example, the confirmation signal is a signal triggered by a user gesture, such as a signal triggered by a user lifting a finger or a signal triggered by a double-click operation after lifting a finger. However, this is only an example, and the confirmation signal is not limited to a signal triggered by a gesture. In the case of a gesture triggered, the gesture is not limited to a gesture such as raising a finger or clicking after the finger is raised.
根据本发明上述实施例的文本输入方法,提高无视觉反馈条件下中英文输入的准确性,以及使用语音与用户交互,非常适于盲人使用。该文本输入方法可以例如应用于智能手机、平板电脑等计算装置上。The text input method according to the above embodiments of the present invention improves the accuracy of Chinese and English input without visual feedback, and uses voice to interact with the user, which is very suitable for blind people to use. The text input method can be applied, for example, to a computing device such as a smart phone or a tablet computer.
图4示出了根据本发明另一实施例的文本输入方法200的总体流程。FIG. 4 shows an overall flow of a text input method 200 according to another embodiment of the present invention.
如图4所示,在步骤S210中,响应于来自触摸传感器的触摸输入信号,确定用户手指在屏幕上的坐标位置。As shown in FIG. 4, in step S210, the coordinate position of the user's finger on the screen is determined in response to a touch input signal from the touch sensor.
在步骤S220中,确定本次触摸位置与前次触摸抬起位置之间距离是否小于某阈值。In step S220, it is determined whether the distance between the current touch position and the previous touch lift position is less than a certain threshold.
接收到用户接触事件后,通过判断本次触摸位置与前次触摸位置是否位置接近,位置接近表明用户接下来要输入的是与上次输入按键相同的字母或者位于其旁边的字符,此时不应对键盘布局进行更新,例如,在两次输入位置基本相同的情况下,则认为本次触摸是在上次抬起时的键盘布局上发生的,且要输入的键是与上次相同的,例如“hello”中,上次输入了“L”,本次触摸位置还是“L”,则认为两次输入相关,无需更新键盘布局,如果更新后,反而可能导致无法正确地输入L;另一种情况是在两次输入位置基本相差一个按键,此时认为盲人用户是想输入上次输入字母相邻的字母,例如,盲人用户想输入“bed”上次输入了“e”,本次触摸位置为上次触摸位置下面的位置,则认为盲人本次欲输入“d”,这是也不应更新键盘布局,同样更新后,反而可能导致无法正确输入。After receiving the user's contact event, by judging whether the current touch position is close to the previous touch position, the close position indicates that the user will next enter the same letter or the character next to the last key input. The keyboard layout should be updated. For example, if the two input positions are basically the same, the current touch is considered to have occurred on the keyboard layout when it was last lifted, and the keys to be input are the same as the last time. For example, in "hello", the last time you entered "L", and the current touch position is still "L", the two inputs are considered to be related, and there is no need to update the keyboard layout. After the update, it may lead to the inability to enter L correctly; another This situation is basically a key difference between the two input positions. At this time, it is considered that the blind user wants to input the adjacent letter of the last entered letter. For example, the blind user wants to enter "bed" and the last time "e" is input. The position is the position below the last touched position, it is considered that the blind person wants to enter "d" this time. This is also the reason that the keyboard layout should not be updated. After the same update, it may be As a result, it cannot be entered correctly.
因此如果步骤S220确定结果为是,即移动范围没有超过一定阈值,则前进到步骤S240,即将键盘布局恢复为默认标准键盘布局,否则步骤S220确定结果为否,即用户手指相对上次点击位置移动超过一定阈值,则前进到步骤S230。Therefore, if the determination result of step S220 is yes, that is, the movement range does not exceed a certain threshold, then proceed to step S240, and restore the keyboard layout to the default standard keyboard layout; otherwise, the determination result of step S220 is no, that is, the user's finger moves relative to the last clicked position. If it exceeds a certain threshold, the process proceeds to step S230.
在步骤S230中,根据触摸位置,计算各字符的输入字符。关于步骤S230的具体实现,可以参考结合图1中的步骤S120进行的描述。In step S230, an input character of each character is calculated based on the touched position. For specific implementation of step S230, reference may be made to the description made in conjunction with step S120 in FIG.
在步骤S250中,根据各字符的输入概率值,确定候选最优字符。In step S250, the candidate optimal character is determined according to the input probability value of each character.
在步骤S260中,视情况尝试更新键盘布局,使得候选最优字符在触摸位置,确定此键盘布局是否符合预定标准。In step S260, an attempt is made to update the keyboard layout as appropriate so that the candidate optimal character is at the touch position, and it is determined whether the keyboard layout meets a predetermined standard.
在所确定的候选最优字符并不是当前键盘布局上手指所触摸位置对应的字符(键)时,尝试更新键盘布局,使得手指位置对应的字符是候选最优字符。When the determined candidate optimal character is not the character (key) corresponding to the position touched by the finger on the current keyboard layout, an attempt is made to update the keyboard layout so that the character corresponding to the finger position is the candidate optimal character.
这里的更新键盘布局包括例如移动和/或压缩键盘。本例通过对软键盘的移动、缩放来调整软键盘与用户的交互,保持用户在调整按键过程中对键盘相对位置的记忆,提高用户调整输入的效率。Updated keyboard layouts here include, for example, mobile and / or compressed keyboards. In this example, the interaction between the soft keyboard and the user is adjusted by moving and zooming the soft keyboard, keeping the user's memory of the relative position of the keyboard during the key adjustment process, and improving the efficiency of the user's input adjustment.
移动和/或压缩键盘可以包括:进行分行或整体移动、压缩;以触摸位置为中心按照线性或指数方式对键盘进行压缩,该线性或指数方式是用户可 选的。Moving and / or compressing the keyboard may include: performing branch or overall movement and compression; compressing the keyboard in a linear or exponential manner with the touch position as the center, and the linear or exponential manner is user selectable.
在系统做出一次更新或调整软键盘的请求后,将判断本次请求是否合法,换句话说,是否符合预定标准。这一判断的判据可以为:移动不得超过预定义的有效范围;压缩不得使任何按键尺寸小于用户能够进行触摸选择的尺寸阈值,例如是否有按键的尺寸小于原尺寸的一定百分比,若在调整后存在这样的按键,则本次请求不合法,不会进行调整。如果请求合法,键盘将实际进行调整,调整方式为键盘整体按照线性缩放、平移,从而使目标按键的中心调整至目标位置。这里,关于移动不得超过预定义的有效范围,可以例如以目前常用输入法的输入范围作为参考来加以确定。After the system makes a request to update or adjust the soft keyboard, it will determine whether the request is legal, in other words, whether it meets the predetermined criteria. The criterion for this judgment can be: the movement must not exceed the predefined effective range; the compression must not make any key size smaller than the size threshold that the user can touch, such as whether any key size is smaller than a certain percentage of the original size. If such a key exists later, the request is illegal and no adjustment will be made. If the request is valid, the keyboard will actually be adjusted. The adjustment method is that the entire keyboard is scaled and translated linearly, so that the center of the target key is adjusted to the target position. Here, the movement must not exceed a predefined effective range, which can be determined by using, for example, an input range of a current commonly used input method as a reference.
如图4所示,在步骤S260中确定键盘布局符合预定标准的情况,前进到步骤S280,即采用更新的键盘布局,确认候选最优字符为最优字符,播放最优字符,否则前进到步骤S270,采用默认的键盘布局,确认在默认的键盘布局下触摸位置对应的按键为最优字符,播放最优字符语音。As shown in FIG. 4, if it is determined in step S260 that the keyboard layout meets the predetermined standard, proceed to step S280, that is, use the updated keyboard layout, confirm that the candidate optimal character is the optimal character, and play the optimal character, otherwise proceed to step In S270, a default keyboard layout is adopted, and the key corresponding to the touch position under the default keyboard layout is determined to be the optimal character, and the optimal character voice is played.
图5a和图5b展示了实际使用中,软键盘移动、缩放的过程示例。具体为键入词汇“朋友”(PENGYOU)时输入第四个拼音字母的场景。实际触摸按键为“F”,算法预测用户意图为键入“G”,算法向键盘申请将G键的中心移动至当前触摸点,键盘确认这一移动合法,最终实现软键盘的移动,同时用户实际得到的语音反馈为“G”。图5b中的字母G上的暗灰色圆形示意手指触摸位置。类似地,图7b、7c中的暗灰色圆形也指示手指触摸位置。Figures 5a and 5b show examples of soft keyboard movement and zooming processes in actual use. This is the scene of entering the fourth pinyin letter when typing the word "PENGYOU". The actual touch key is "F", the algorithm predicts that the user intends to type "G", and the algorithm requests the keyboard to move the center of the G key to the current touch point. The keyboard confirms that this movement is legal, and finally realizes the movement of the soft keyboard. The resulting voice feedback is "G". The dark gray circle on the letter G in FIG. 5b indicates the finger touch position. Similarly, the dark gray circles in Figs. 7b, 7c also indicate the finger touch position.
图6示出了实际使用中,软键盘移动非法的一个示意性示例。具体地,输入companies最后一个字母s时,由于按键位置太靠左,压缩按键将导致a键宽度小于最小值(例如,原键宽度的50%),因此移动不合法。虽然概率最大值按键位为s,但由于移动不合法,最终输入按键为按下位置所对应的按键a。FIG. 6 shows a schematic example of illegal movement of a soft keyboard in actual use. Specifically, when the last letter s of the company is input, since the key position is too left, compressing the key will cause the a key width to be smaller than the minimum value (for example, 50% of the original key width), so the movement is illegal. Although the probability maximum key bit is s, because the movement is illegal, the final input key is the key a corresponding to the pressed position.
回到图4,在步骤S290中,持续监测事件,如果接收到移动事件,移动过程中给出语音反馈。例如,在系统给出最优字符的语音提示后,用户认为这不是自己想要的字符,此时手指会继续移动以寻找其余的字符,此过程中,会持续给出语音反馈。Returning to FIG. 4, in step S290, the event is continuously monitored. If a mobile event is received, voice feedback is given during the movement. For example, after the system gives a voice prompt for the optimal character, the user thinks that this is not the character he wants, at this time the finger will continue to move to find the remaining characters, and in the process, the voice feedback will continue to be given.
在步骤S291中,响应于接收到用户抬起事件,进行手势判断,给出语 音提示。In step S291, in response to receiving the user lifting event, a gesture judgment is performed, and a voice prompt is given.
在一个示例中,通过手势操作实现删除、确认选词、切换候选词、清空等操作。In one example, operations such as deleting, confirming word selection, switching candidate words, and clearing are implemented through gesture operations.
确定是否发生了手势输入可以包括:在全屏范围内而非局限于软键盘所占据区域检测触点轨迹;利用触点轨迹的位移-时间关系判断是否发生手势输入并判断手势类型,从而能基于判定的手势类型,而确定对应的输入操作。Determining whether a gesture input has occurred includes detecting contact trajectories in a full screen range instead of being limited to the area occupied by the soft keyboard; using the displacement-time relationship of the contact trajectories to determine whether gesture input has occurred and determining the type of gesture, so that the Type of gesture while determining the corresponding input operation.
在一个示例中,通过以下步骤判断用户是否发生了手势操作:用户完成抬起动作时,如果整个动作的持续时间小于例如200ms,且在垂直或水平方向上的位移大于例如70像素,且大于另一方向上的位移,则判定用户进行了手势操作。需要说明的是,这里的手势操作不包括手指在触摸状态下移动的操作,即不包括手指在屏幕上搜索的动作。单指或者双指操作可以由接触点来进行判定。In one example, it is determined whether the user has performed a gesture operation through the following steps: When the user completes the lifting motion, if the duration of the entire motion is less than, for example, 200ms, and the displacement in the vertical or horizontal direction is greater than, for example, 70 pixels, and greater than another The displacement in one direction determines that the user has performed a gesture operation. It should be noted that the gesture operation here does not include the operation of the finger moving in the touch state, that is, it does not include the action of searching for the finger on the screen. One-finger or two-finger operation can be determined by the touch point.
在一个示例中,手势类型可以包括:单指左划、双指左划、单指上划、单指下划、单指右划等。In one example, the gesture type may include: single-finger swipe left, two-finger swipe left, single-finger swipe up, single-finger swipe down, single-finger swipe right, and so on.
示例性地,手势与操作的对应关系可以如下:Exemplarily, the correspondence between gestures and operations can be as follows:
单指左划:中文多步选词时撤销上一步选词,其他情况下删除当前没有确认选词的最后一个拼音或者字母,如果没有则删除最后一个完成选词的汉字或者字母。Single-finger left stroke: In the Chinese multi-step word selection, the previous word selection is canceled. In other cases, the last pinyin or letter of the currently selected word is not confirmed. If not, the last Chinese character or letter that completed the word selection is deleted.
单指右划:对当前候选词确认输入,中文多步选词完成后自动上屏。Single-finger right swipe: Confirm the input of the current candidate word, and the screen will be automatically displayed after the Chinese multi-step word selection is completed.
单指上划:设置当前高亮的候选词为列表里的上一个候选词。One-finger swipe up: Set the currently highlighted candidate as the previous candidate in the list.
单指下划:设置当前高亮的候选词为列表里的下一个候选词。One-finger swipe down: Set the currently highlighted candidate as the next candidate in the list.
双指左划:清空当前所有的输入。Two-finger swipe left: Clear all current input.
在一个示例中,在判断出手势类型后,给出相应地语音反馈,例如在判断为双指左划手势时,语音提示“清空当前所有的输入”。In one example, after the gesture type is determined, corresponding voice feedback is given, for example, when it is determined that the two-finger left-swipe gesture is performed, the voice prompt "clear all current inputs".
在步骤S292中,进行所判断的手势动作对应的操作,在确定为输入操作的清空下,还记录下抬起动作的相关信息。In step S292, the operation corresponding to the determined gesture action is performed, and the information related to the lifting action is also recorded under the clearing determined as the input operation.
例如,在确定用户为进行了对应于输入已音频提示的字符的抬起动作时,输入该字符,并且记录本次抬起的相关信息,用于下一次输入的比较。For example, when it is determined that the user has performed a lifting action corresponding to the input of a character with audio prompts, the character is input, and the related information of the current lifting is recorded for comparison of the next input.
后续当收到用户的触摸事件时,再次从步骤S210开始执行。When a subsequent touch event is received from the user, the process starts from step S210 again.
根据本发明上述实施例的文本输入方法,结合语言模型与落点模型对用户输入进行预测,通过移动、缩放软键盘的方式适应无视觉反馈的输入,使用语音与用户交互,使用手势操作完成文字输入中的选词、确认、删除、清空等操作,能够提高无视觉反馈条件下中英文输入的准确性,以及提高无视觉反馈条件下中英文输入的速度。According to the text input method of the above embodiment of the present invention, the user input is predicted by combining the language model and the drop point model, and the input without visual feedback is adapted by moving and zooming the soft keyboard, using voice to interact with the user, and using gesture operations to complete the text Word selection, confirmation, deletion, and clearing operations during input can improve the accuracy of Chinese and English input without visual feedback and the speed of Chinese and English input without visual feedback.
需要说明的是,键盘布局并不限于是视觉可见的键盘布局,也可以是不可见的能够达到相同效果的键盘响应区域设置,这对于盲人来说效果是相同的。It should be noted that the keyboard layout is not limited to a visually visible keyboard layout, and may also be an invisible keyboard response area setting capable of achieving the same effect, which has the same effect for blind people.
下面以输入hello为例,给出利用本发明实施例的文本输入方法输入一个单词的完整过程。Taking the input hello as an example, a complete process of inputting a word using the text input method of the embodiment of the present invention is given below.
图7a示出了用户按下按键前键盘状态。Figure 7a shows the keyboard state before the user presses a key.
1、用户在键盘区域进行按下操作后,将利用语言模型和落点模型计算用户期望按下的概率最大的键,并通过缩放、平移键盘,将该键的中心移至用户触点。同时,语音反馈内容为当前按键,如‘H’。(键盘上通过角标显示预测顺序,为调试、展示使用,非反馈内容)1. After the user presses in the keyboard area, the language model and the drop point model will be used to calculate the key with the highest probability that the user expects to press, and the center of the key will be moved to the user touch point by zooming and panning the keyboard. At the same time, the voice feedback content is the current key, such as 'H'. (The prediction order is displayed on the keyboard through the corner mark, for debugging and display use, non-feedback content)
图7b示出了用户在输入一个字母时,首次按下后键盘的布局。Figure 7b shows the layout of the keyboard after the user first presses a letter when entering a letter.
2、若这一按键并非用户期望输入的按键,则可在保持手指按下的状态下,通过移动手指检索按键。检索过程中将实时通过语音反馈用户手指经过的按键,正在进行的语音反馈会被新的语音所中断。检索过程中,键盘的布局保持。2. If the key is not the key that the user desires to input, the key can be retrieved by moving the finger while keeping the finger pressed. During the retrieval process, the keys passed by the user's finger will be feedback in real time through voice, and the ongoing voice feedback will be interrupted by the new voice. During retrieval, the keyboard layout is maintained.
3、当用户通过语音反馈确认自己要输入的按键后,通过抬起手指的方式确认输入。同时,语音反馈用户输入的按键,如‘H’。用户键入后,键盘的布局保持。3. After the user confirms the key they want to input through voice feedback, confirm the input by raising their finger. At the same time, the keys input by the user, such as 'H', are voice feedback. After the user types, the keyboard layout remains.
4、重复1-3的过程,进行‘Hello’剩余字母的输入。4. Repeat the process from 1-3 to enter the remaining letters of 'Hello'.
5、用户在单词输入过程中可以使用手势删除已输入的字母。通过左划操作,可以删除最后输入的字母。同时语音反馈:“X已删除”(X为被删除的字母)。5. The user can use gestures to delete the entered letters during word input. With the left stroke operation, you can delete the last letter entered. At the same time, the voice feedback: "X has been deleted" (X is the deleted letter).
图7c示出了输入“o”时发生错误的情形,示例中,错误输入了“i”,使用左划手势删除,此时语音反馈:“i已删除”。Fig. 7c shows a situation where an error occurs when "o" is input. In the example, "i" is input by mistake and deleted with a left swipe gesture. At this time, the voice feedback: "i has been deleted".
这里结合图7c说明一下该示例中屏幕显示的内容,不过需要注意,此仅 为示例,而且此主要为验证实验结果进行的显示,而在实际应用于盲人输入服务的情况下,屏幕上可以什么都不显示,可以显示部分内容,也可以显示不同的内容。图7c中,横线引出的标号1指示根据本发明实施例的当前输入的内容也即用户确认输入的内容,本示例中为通过用户抬起动作确认的输入,标号2指示根据本发明实施例的算法预测的内容,标号3指示在不运行本发明实施例的情况下的输入的内容,也即按照默认布局,触摸落点所输入的内容。标号4指示当前输入的字符为i,标号5指示候选单词,包括helli,hellish,helling和hellip,另外,这里下面显示的软键盘的字符上显示的数字表示通过概率计算得出的输入概率排名前五的字符,即该示例中输入概率第一的字符为I,输入概率第二的字符为O,输入概率第三的字符为U,输入概率第四的字符为J,输入概率第五的字符为Y。Here, the content displayed on the screen in this example will be described with reference to FIG. 7c, but it should be noted that this is only an example, and this is mainly a display of verification experiment results, and what can be done on the screen when it is actually applied to a blind input service Neither display, you can display some content, or you can display different content. In FIG. 7c, the reference numeral 1 drawn by the horizontal line indicates the content currently input according to the embodiment of the present invention, that is, the content confirmed by the user. In this example, the input is confirmed by the user's lifting action, and the reference numeral 2 indicates according to the embodiment of the present invention. The content predicted by the algorithm, reference numeral 3 indicates the input content without running the embodiment of the present invention, that is, the content input by touching the touch point according to the default layout. The number 4 indicates that the currently entered character is i, and the number 5 indicates candidate words, including heli, hellish, helling, and hellip. In addition, the numbers displayed on the characters of the soft keyboard shown below indicate the input probability calculated by the probability ranking. Five characters, that is, the character with the first probability of input is I, the character with the second probability of input is O, the character with the third probability is U, the character with the fourth probability is J, and the character with the fifth probability Is Y.
6、当用户完成一个单词的输入时,使用右划手势确认输入单词。同时,语音反馈“Hello已输入”。6. When the user finishes inputting a word, use the right swipe gesture to confirm the input word. At the same time, the voice feedback "Hello has been entered".
另外,在没有输入字母的情况下,右划操作会键入一个空格,同时,语音反馈“空格”。In addition, if no letter is entered, the right swipe operation will type a space, and at the same time, the voice feedback "space".
另外,用户使用上划、下划手势可以选择上一个、下一个候选词,同时,语音反馈改候选词。这一功能在英文输入中用处较小,主要在中文输入有同音词时使用。In addition, the user can use the up and down gestures to select the previous and next candidate words, and at the same time, the voice feedback changes the candidate words. This function is less useful in English input, mainly used when there are homophones in Chinese input.
另外,用户使用双指左划手势可以清空所有输入的内容。同时,语音反馈“清空”。In addition, the user can use a two-finger left swipe gesture to clear all input. At the same time, the voice feedback "empties".
另外,用户使用手指左划手势,可以删除刚输入的字符,例如图7d示意性地示出了使用左划手势删除“o”,此时语音反馈:“o已删除”,图7d中的圆圈和向左的箭头示意了手指左划的手势。In addition, the user can use the left swipe gesture of the finger to delete the character just entered. For example, FIG. 7d schematically illustrates the deletion of “o” using the left swipe gesture. At this time, the voice feedback: “o has been deleted”, the circle in FIG. 7d And the left arrow gestures to the left.
本实施例的输入方法特别适合于盲人使用,不过当然也可以由视力正常的人。The input method of this embodiment is particularly suitable for use by blind people, but of course it can also be used by people with normal vision.
需要说明的是,前面的各个手势和对应的操作仅为示例,可以视需要设计其它的手势以及与不同的操作对应。It should be noted that the previous gestures and corresponding operations are merely examples, and other gestures may be designed and corresponding to different operations as required.
本发明实施例的输入方法可以应用于各种计算装置,例如智能手机、平板电脑等等。The input method of the embodiment of the present invention can be applied to various computing devices, such as a smart phone, a tablet computer, and the like.
本发明实施例还可以实施为计算装置,包括:触摸屏,具有触摸传感 器,能够检测触摸输入;控制器和存储器,存储器上存储有计算机可执行指令,在触摸式软键盘已启动的情况下:当所述计算机可执行指令当被控制器执行时,可操作来执行上述输入方法。Embodiments of the present invention may also be implemented as a computing device, including: a touch screen with a touch sensor capable of detecting a touch input; a controller and a memory, the computer-executable instructions stored on the memory, and when the touch soft keyboard is activated: The computer-executable instructions, when executed by a controller, are operable to perform the input method described above.
本发明实施例还可以实现为计算机可读存储介质,其上存储有计算机可执行指令,在计算装置的触摸式软键盘已启动的情况下:当所述计算机可执行指令当被计算机执行时,可操作来执行上述输入方法。Embodiments of the present invention may also be implemented as a computer-readable storage medium having computer-executable instructions stored thereon. When the touch-type soft keyboard of a computing device has been activated: when the computer-executable instructions are executed by a computer, Operable to perform the input method described above.
以上已经描述了本发明的各实施例,上述说明是示例性的,并非穷尽性的,并且也不限于所披露的各实施例。在不偏离所说明的各实施例的范围和精神的情况下,对于本技术领域的普通技术人员来说许多修改和变更都是显而易见的。因此,本发明的保护范围应该以权利要求的保护范围为准。The embodiments of the present invention have been described above, the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the various embodiments described. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims (16)

  1. 一种计算装置中利用触摸式软键盘进行输入的输入方法,包括:An input method using a touch-type soft keyboard in a computing device includes:
    响应于来自触摸传感器的触摸输入信号,确定用户手指在屏幕上的坐标位置;Determining the coordinate position of the user's finger on the screen in response to a touch input signal from the touch sensor;
    根据触摸按下位置,计算各字符的输入概率;Calculate the input probability of each character according to the touched position;
    根据各字符的输入概率值,确定最优字符;以及Determine the optimal character based on the input probability value of each character; and
    播放该最优字符的语音。Play the voice of the best character.
  2. 根据权利要求1所述的输入方法,所述确定最优字符包括:The input method according to claim 1, the determining an optimal character comprises:
    根据各字符的输入概率值,确定概率值最大的字符作为候选最优字符;Determine the character with the highest probability value as the candidate optimal character according to the input probability value of each character;
    根据触摸按下位置和确定的候选最优字符,尝试更新键盘布局,以使得所述候选最优字符在所述触摸按下位置且更新后的键盘布局满足预定标准;Try to update the keyboard layout according to the touch-down position and the determined candidate optimal character, so that the candidate optimal character is at the touch-down position and the updated keyboard layout meets a predetermined standard;
    如果更新键盘布局成功,则确定候选最优字符为最优字符;If the keyboard layout is updated successfully, determine the candidate optimal character as the optimal character;
    否则采用默认标准键盘布局,确定最优字符为该布局下的触摸按下位置的按键。Otherwise, the default standard keyboard layout is used, and the optimal character is determined as the key at the touched position in this layout.
  3. 根据权利要求2所述的输入方法,还包括:The input method according to claim 2, further comprising:
    响应于用户不抬起手指继续移动访问其他按键字符,播放手指位置所对应按键字符的语音。In response to the user continuing to access other key characters without lifting the finger, the voice of the key character corresponding to the finger position is played.
  4. 根据权利要求1所述的输入方法,还包括:The input method according to claim 1, further comprising:
    在播放按键字符的语音之后,响应于检测到源于用户的确认信号,而完成被确认的字符的输入。After playing the voice of the key character, the input of the confirmed character is completed in response to detecting a confirmation signal from the user.
  5. 根据权利要求4所述的输入方法,所述确认信号包括:The input method according to claim 4, the confirmation signal comprises:
    用户抬起手指触发的信号或者抬起手指后双击点击操作触发的信号。The signal triggered by the user lifting the finger or the signal triggered by a double-click operation after lifting the finger.
  6. 根据权利要求1所述的输入方法,所述确定各字符的输入概率包括:The input method according to claim 1, wherein determining the input probability of each character comprises:
    基于已经输入的字符串和用户触摸按下位置,结合使用落点模型与语言模型,计算用户当前输入各字符的概率。Calculate the probability of each character currently entered by the user based on the character string that has been entered and the position that the user touched and pressed, using a combination of the landing point model and the language model.
  7. 根据权利要求1所述的输入方法,在所述根据触摸位置计算各字符的输入概率之前,还包括:The input method according to claim 1, before the calculating the input probability of each character according to the touch position, further comprising:
    确定本次触摸位置与前次确认输入的触摸位置之间的距离是否小于一定阈值,Determine whether the distance between the current touch position and the previous confirmed touch input position is less than a certain threshold,
    如果是,则直接使用前次确认输入时的键盘布局,不对键盘布局做更新;If so, the keyboard layout at the time of the previous confirmation input is directly used, and the keyboard layout is not updated;
    如果否,则将键盘布局恢复为默认标准键盘布局,然后再做概率计算和确定最优字符,并根据该结果视需要更新键盘布局。If not, restore the keyboard layout to the default standard keyboard layout, then do probability calculations and determine the optimal characters, and update the keyboard layout as needed based on the results.
  8. 根据权利要求2-7所述的输入方法,所述键盘布局为不可见的能够达到相同效果的键盘响应区域设置。The input method according to claim 2-7, wherein the keyboard layout is an invisible keyboard response area setting capable of achieving the same effect.
  9. 根据权利要求7诉述的输入方法,所述落点模型为正态分布。The input method according to claim 7, wherein the landing point model is a normal distribution.
  10. 根据权利要求2所述的输入方法,所述更新键盘布局包括移动和/或压缩键盘。The input method according to claim 2, wherein said updating the keyboard layout comprises moving and / or compressing the keyboard.
  11. 根据权利要求11所述的输入方法,所述移动和/或压缩键盘包括:The input method according to claim 11, the mobile and / or compressed keyboard comprises:
    进行分行或整体移动、压缩;Perform branch or overall movement and compression;
    以触摸按下位置为中心按照线性或指数方式对键盘进行压缩;Compress the keyboard linearly or exponentially with the touched position as the center;
    移动不得超过预定义的有效范围;Movement must not exceed a predefined valid range;
    压缩不得使任何按键尺寸小于尺寸阈值。Compression must not make any key size smaller than the size threshold.
  12. 根据权利要求1所述的输入方法,还包括确定是否发生了手势输入,The input method according to claim 1, further comprising determining whether a gesture input has occurred,
    所述确定是否发生了手势输入包括:The determining whether a gesture input has occurred includes:
    在全屏范围内而非局限于软键盘所占据区域检测触点轨迹;Detect contact trajectories in the full screen range, not limited to the area occupied by the soft keyboard;
    利用触点轨迹的位移-时间关系判断是否发生手势输入并判断手势类型;Use the displacement-time relationship of the contact trajectory to determine whether gesture input has occurred and determine the type of gesture;
    基于判定的手势类型,而确定对应的输入操作。A corresponding input operation is determined based on the determined gesture type.
  13. 根据权利要求13的输入方法,所述手势类型以及对应的输入操作包括:The input method according to claim 13, the gesture types and corresponding input operations include:
    单指左划:删除最后键入的字符;Single-finger swipe left: delete the last character typed;
    双指左划:清空所有键入的字符;Two-finger swipe left: Clear all typed characters;
    单指上划:将高亮的候选词切换至上一个候选词;Single-finger swipe up: Switch the highlighted candidate to the previous candidate;
    单指下划:将高亮的候选词切换至下一个候选词;Single finger swipe down: Switch the highlighted candidate to the next candidate;
    单指右划:确认输入当前高亮的候选词One-finger right swipe: Confirm input of currently highlighted candidate
  14. 根据权利要求1到14任一项所述的输入方法,还包括:The input method according to any one of claims 1 to 14, further comprising:
    对于触摸操作提供音频反馈,包括:Provide audio feedback for touch operations, including:
    对按键按下的反馈;Feedback on key presses;
    对确认输入的反馈;Feedback on confirmation inputs;
    对手势操作的反聩。In response to gestures.
  15. 一种计算装置,包括:A computing device includes:
    触摸屏,具有触摸传感器,能够检测触摸输入;Touch screen with touch sensor capable of detecting touch input;
    控制器和存储器,存储器上存储有计算机可执行指令,在触摸式软键盘已启动的情况下:当所述计算机可执行指令当被控制器执行时,可操作来执行权利要求1到15所述的输入方法。A controller and a memory, where computer-executable instructions are stored on the memory, and when the touch soft keyboard is activated: when the computer-executable instructions are executed by the controller, they are operable to execute the claims 1 to 15 Input method.
  16. 一种计算机可读存储介质,其上存储有计算机可执行指令,在计算装置的触摸式软键盘已启动的情况下:当所述计算机可执行指令当被计算机执行时,可操作来执行权利要求1到15所述的输入方法。A computer-readable storage medium having computer-executable instructions stored thereon, in a case where a touch soft keyboard of a computing device has been activated: when the computer-executable instructions are executed by a computer, they are operable to execute claims Input method described in 1 to 15.
PCT/CN2018/094203 2018-07-03 2018-07-03 Blind input method using touch soft keyboard, computing apparatus, and storage medium WO2020006676A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2018/094203 WO2020006676A1 (en) 2018-07-03 2018-07-03 Blind input method using touch soft keyboard, computing apparatus, and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2018/094203 WO2020006676A1 (en) 2018-07-03 2018-07-03 Blind input method using touch soft keyboard, computing apparatus, and storage medium

Publications (1)

Publication Number Publication Date
WO2020006676A1 true WO2020006676A1 (en) 2020-01-09

Family

ID=69059835

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/094203 WO2020006676A1 (en) 2018-07-03 2018-07-03 Blind input method using touch soft keyboard, computing apparatus, and storage medium

Country Status (1)

Country Link
WO (1) WO2020006676A1 (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101719022A (en) * 2010-01-05 2010-06-02 汉王科技股份有限公司 Character input method for all-purpose keyboard and processing device thereof
CN101937313A (en) * 2010-09-13 2011-01-05 中兴通讯股份有限公司 Dynamic generation and input method and device for touch keyboard
CN102902351A (en) * 2011-07-25 2013-01-30 富泰华工业(深圳)有限公司 Touch electronic device

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101719022A (en) * 2010-01-05 2010-06-02 汉王科技股份有限公司 Character input method for all-purpose keyboard and processing device thereof
CN101937313A (en) * 2010-09-13 2011-01-05 中兴通讯股份有限公司 Dynamic generation and input method and device for touch keyboard
CN102902351A (en) * 2011-07-25 2013-01-30 富泰华工业(深圳)有限公司 Touch electronic device

Similar Documents

Publication Publication Date Title
US11868609B2 (en) Dynamic soft keyboard
US10140284B2 (en) Partial gesture text entry
US9851809B2 (en) User interface control using a keyboard
KR101366723B1 (en) Method and system for inputting multi-touch characters
US9158757B2 (en) System and method for preview and selection of words
US9411508B2 (en) Continuous handwriting UI
US10838513B2 (en) Responding to selection of a displayed character string
KR101602840B1 (en) Smart user-customized virtual keyboard
US20110296333A1 (en) User interaction gestures with virtual keyboard
US20140198047A1 (en) Reducing error rates for touch based keyboards
US20100315266A1 (en) Predictive interfaces with usability constraints
EP2575009A2 (en) User interface method for a portable terminal
US20180046224A1 (en) Input Apparatus, Information Processing Method, And Information Processing Apparatus
WO2020006676A1 (en) Blind input method using touch soft keyboard, computing apparatus, and storage medium
CN111367459B (en) Text input method using pressure touch pad and intelligent electronic device
US9261973B2 (en) Method and system for previewing characters based on finger position on keyboard
EP2811371B1 (en) Method and system for previewing characters based on finger position on keyboard

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18925401

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18925401

Country of ref document: EP

Kind code of ref document: A1