CN107643824B

CN107643824B - System and method for braille touch reading by left-right sliding gesture

Info

Publication number: CN107643824B
Application number: CN201710726752.XA
Authority: CN
Inventors: 褚少微; 张帆; 朱科颖; 张方圆; 吉娜烨
Original assignee: Zhejiang University of Media and Communications
Current assignee: Zhejiang University of Media and Communications
Priority date: 2017-08-22
Filing date: 2017-08-22
Publication date: 2021-07-09
Anticipated expiration: 2037-08-22
Also published as: CN107643824A

Abstract

The invention discloses a left-right sliding gesture Braille touch reading system and method. Different vibration tactile feedbacks are arranged on the touch screen according to different sliding directions of fingers, and the touch reading of the braille is realized by continuously sliding the fingers for six times from left to right and sequentially touching and reading the information of six concave-convex points in one braille; the system comprises a touch screen, a Braille conversion module, a control module and a vibration module. The touch screen device solves the technical problem of touch text reading of the touch screen device, has the advantages of high touch reading speed, simplicity in operation and easiness in learning and use, and can be conveniently realized on various touch screen devices such as smart phones and tablet computers.

Description

System and method for braille touch reading by left-right sliding gesture

Technical Field

The invention relates to a left-right sliding gesture braille touch reading system and method, and belongs to the technical field of touch interaction of touch screen equipment.

Background

Vibrotactile feedback of smart mobile devices is one of the key technologies to achieve non-visual information acquisition and blind interaction. Tactile feedback may provide information such as: message reminding, emotion transfer, non-visual interaction, and reading text in blind situations. Especially for blind users, touch interaction is a key means for using mobile devices such as mobile phones and the like to touch screens; and is the most important method for realizing electronic reading of braille.

How to design the vibration touch feeling on the touch screen mobile device enables the blind person to use the touch screen mobile device such as a smart phone and the like is an important research direction of human-computer interaction at present. And is one of the key problems for helping the blind people to integrate into the normal society, enjoy the information technology development results and improve the life quality.

The main method for realizing touch character reading on the touch screen mobile device at present is to encode the vibration touch feedback and simulate the concave-convex or Morse code of six-point Braille. The methods of tactile sensing can be classified into three categories according to the way the finger interacts and the type of vibration feedback.

The first method is that concave-convex information of 6 braille points is sequentially expressed by combining different tactile feedback patterns sensed by fingers in the sliding process on a touch screen; the user then identifies the touched braille based on the sequence of tactile feedback. The method separates the six Braille points, and then obtains the concave-convex information of each Braille point in sequence to realize the reading purpose of the Braille, thereby achieving better effect. However, such a method is limited in that it requires a finger to slide a long distance on a touch screen and sequentially senses the concave-convex information of six braille points according to the slid distance. In the using process, a user needs to pay high attention so as to avoid missing concave-convex information of a certain Braille point in the tactile feedback sequence. This makes such methods tend to be slow to touch, typically requiring more than 5 seconds to touch a braille.

The second method is to express text information by encoding the vibration rhythm of the touch screen mobile device. For example, the opening and closing of the vibration are encoded in time series to express the unevenness of six points of braille in order; or simulating the drop and answer codes of the Morse code through different vibration time lengths and interval time. The method has the advantages that by encoding the vibration in time sequence, the time for expressing each literal character is short, and the encoding efficiency is high. Typically the speed of presentation of textual tactile codes can be up to 855 milliseconds per character. However, because the prosodic codes have short tactile patterns, accurate identification is often difficult. The reading accuracy of the method is lower and is only about 71%. And the rhythm-based method is passive tactile perception, the controllability is poor, the learning and memory burden is large in the operation process, and the user experience is often poor.

A third type of method is spatial haptic coding. The screen of the touch screen mobile device is divided into regions, and the regions correspond to the spatial positions of six-point braille one by one. And judging the concave-convex of the corresponding Braille point according to different vibration feedbacks by touching different areas with fingers, thereby achieving the purpose of reading Braille. Due to the advantage that the designed space touch position corresponds to the six-point Braille points one to one, the scheme is easy to learn and master. However, the biggest problem of this method is that it requires the finger to slide to six different areas to read the tactile patterns, so that the character reading efficiency is low, and it takes 4-26 seconds to read one character on average.

Disclosure of Invention

In order to solve the problems in the background art, the invention provides a left-right sliding gesture touch braille touch reading system and a method. The Braille touch reading method and the device combine the application of gesture sliding on the touch screen and vibration tactile feedback of the device to touch and read the Braille, solve the technical problem of touch and reading of characters for the blind in the background technology, and provide a solution for blind users to use touch screen intelligent devices such as mobile phones and the like and electronic Braille reading.

The method for solving the problem is that the semantic information of six concave-convex points in the blind text is sequentially switched by using the gesture of sliding the finger left and right on the touch screen intelligent device, wherein the sliding direction is changed each time. Therefore, touching and reading each braille character requires the finger to continuously slide on the touch screen left and right six times, and whether the read braille dot is convex or concave is judged according to the existence or nonexistence of tactile feedback each time.

The specific technical scheme is as follows:

a touch Braille touch reading system comprises:

the method comprises the steps that a touch screen is used for detecting a finger touch position and sending the finger touch position and time sequence change information to a control module;

the Braille conversion module reads character information from the intelligent equipment, converts the character information into six-point Braille, converts concave-convex information of the six-point Braille into Braille signals and sends the Braille signals to the control module.

The six-point Braille is six concave-convex points of standard Braille.

The Braille touch control system comprises a control module, a vibration module and a display module, wherein the control module judges the sliding direction of a finger according to the touch position of the finger received from a touch screen and the time sequence change information of the finger, generates a vibration control signal according to the sliding direction of the finger and a Braille signal received from a Braille conversion module, and sends the vibration control signal to the vibration module to control the generation of vibration;

the device comprises a vibration module, wherein the vibration module is connected to a touch screen or directly connected to a finger and is fed back to the finger through vibration.

The finger sliding direction is divided into a left direction and a right direction along the surface of the touch screen, and the control module judges whether the finger sliding direction is the left direction or the right direction according to the finger touch position and the change information of the finger touch position in the time sequence.

The control module generates a vibration control signal according to the sliding direction of the fingers and the information of the concave-convex surface of the Braille point currently presented in the Braille signals, and sends the vibration control signal to the vibration module to enable the vibration module to vibrate the fingers, and the Braille concave-convex touch feeling corresponding to the touch feeling area of the vertical bar is fed back.

The touch screen, the Braille conversion module, the control module and the vibration module are all integrated in the integrated equipment.

Secondly, a touch Braille touch reading method:

and a finger slides on the touch screen, the touch screen provides vibration according to the presented Braille information, and the Braille is touched and read by sensing the vibration through sliding the finger on the whole touch screen to obtain feedback.

The method comprises the steps that a finger slides on a touch screen in a one-way mode to represent touch reading of one Braille point in six Braille points, the touch screen vibrates correspondingly to represent the touch reading Braille point to be a convex surface, and the touch screen does not vibrate correspondingly to represent the touch reading Braille point to be a concave surface.

The process of touching and reading the braille is as follows: and touching and sliding a finger from one side edge to the other side edge of the touch screen, touching and reading the concave-convex information of one Braille point in the six Braille points through the touch screen according to the vibration feedback, and repeating the unidirectional sliding operation for six times to complete the touching and reading of all six Braille points in the six Braille points.

And in the process of repeating the unidirectional sliding operation for six times, the sliding direction of the next time is opposite to the sliding direction of the last time. Specifically, the braille touch reading can be performed by touching and reading six braille points of one braille from left sliding to right sliding for six times (sequentially from left to right, from left to right.

In the process of reading one Braille, the presenting sequence of six Braille points is from top to bottom according to rows, and the Braille points are read from left to right in each row, namely, the Braille points are read from the left of the first row and sequentially read from the left of each row from top to bottom.

The specific process of sliding and touching the braille on the touch screen from left to right is that firstly, a finger is pressed down at any point position on the screen and then the screen slides leftwards, if the first blind point corresponding to the braille presented by the current equipment is convex, the mobile phone has vibration touch feedback; and if the corresponding blind spot is concave, the mobile phone does not feed back vibration, and the user operation is marked as (a) at the moment, and the information of the first blind spot is sensed.

Then, the finger slides in the opposite direction (rightward (b)) to touch and read the concave-convex information of the second blind spot. If the corresponding Braille points are convex, the mobile phone has vibration tactile feedback; if the corresponding blind spot is concave, the mobile phone does not feed back vibration, so that the information of the second Braille point is touched and read. By analogy, the finger slides back and forth for 6 times (left (a), right (b), left (c), right (d), left (e) and right (f)) on the screen to complete the touch reading of one Braille.

The distance threshold value of sliding of the finger in different directions in the touch reading process is set to be n pixels (n is larger than or equal to 4), namely the gesture of sliding in the direction is judged when the finger continuously slides in the same direction and exceeds the n pixels, and the mobile phone immediately responds to the existence of vibration touch to feed back to a user. Therefore, the user can slide the finger for a short distance to sense the touch feeling and quickly slide the finger back and forth left and right to read the touch feeling of the braille.

The provision of vibration feedback provides vibration in an intermittent manner, characterized by low noise. Specifically, short vibration time and vibration interval time are set, and the vibration time and the vibration interval time are alternated. The vibration modes formed were: vibration time w1, vibration interval time w2, vibration time w1, vibration interval time w2, … …, whereby vibration time w1 and vibration interval time w2 are repeated continuously to provide intermittent vibration.

The invention has the beneficial effects that:

1) according to the Braille touch reading method and device, Braille touch reading is achieved through touch screen sliding gestures and vibration tactile feedback.

2) In the interaction process, the touch reading of one Braille can be finished by sliding the finger for six times, and the operation is simple and quick.

3) The technology of the invention can be conveniently realized on various touch screen devices provided with vibration sensors: such as smart phones, tablet computers, etc.

The method can start to slide the finger at any position on the touch screen except the left edge, and overcomes the problem that the traditional method needs to sense the space position on the touch screen. In the touch reading process, the fingers only need to slide back and forth left and right within a short range, so that the required touch screen space can be very small, and the touch screen can be used as long as the screen space with the width of two forefingers can be accommodated.

The method can greatly improve the touch and read efficiency, the system immediately feeds back the touch or not in the sliding direction every time the sliding direction is switched, and a skilled user can quickly sense and quickly switch the direction of the sliding of the finger left and right to complete touch and read. The problem of efficiency and ease of use of braille touch reading on touch screen smart devices such as mobile phones is effectively solved.

According to the method, the actual test of 8 users shows that the reading speed can reach 1.5 seconds at the highest speed, and the method can be used proficiently through 2 minutes of practice.

Drawings

FIG. 1 is a schematic numbering drawing of six concave-convex points of Braille in the invention.

FIG. 2 is a schematic diagram showing the sequence of sequentially touching and reading the information of six concave and convex points of Braille by sliding the finger left and right in the invention.

Fig. 3 is a schematic view illustrating a process of reading braille using the braille character r as an example in the present invention.

FIG. 4 is a schematic view of the touch reading process of the Braille character r of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail below with reference to the accompanying drawings and embodiments.

The specific implementation of the invention comprises:

1) the device provides vibration feedback according to the currently presented braille information when the finger clicks the screen and slides, without any graphical image information being presented on the touch screen of the device.

2) When the Braille point information corresponding to the Braille currently presented by the equipment is convex in the leftward or rightward sliding process of the finger, the equipment can provide intermittent vibration feedback; conversely, if the braille point information corresponding to the braille presented by the system is concave, the device will provide intermittent vibration feedback. The tactile feedback mode of vibration feedback is as follows: vibration time w1, vibration interval time w2, vibration time w1, vibration interval time w2, … …, and the vibration and vibration interval time of w1 and w2 are repeated continuously.

The value range of w1 obtained by user tests can be any value between 10 ms and 30ms, the value range of w2 can be any value between 50ms and 150ms, and the generated noise is low. This vibration pattern is only perceptible by the hand-held device and cannot be fed back by visual or audible vibration.

The specific implementation process of reading one Braille is as follows:

1) and clicking any position on the screen by a finger to start to touch and read the braille. The position to start clicking cannot be the leftmost edge of the screen, since the leftmost edge of the screen cannot slide the finger to the left any more;

2) the finger slides to the left to touch and read the first braille point of the braille. If the equipment has vibration feedback at the moment, the first Braille point representing the Braille is convex; on the contrary, if the equipment does not have vibration feedback at the moment, the first Braille point representing the Braille is concave;

3) and after the information of the first Braille point is sensed, the finger slides to the right to touch and read the concave-convex information of the second Braille point. If the vibration feedback is provided for the touch screen through the vibration module at the moment, the second Braille point representing the Braille is convex; on the contrary, if the vibration feedback is not provided for the touch screen through the vibration module at the moment, the second Braille point representing the Braille is concave;

4) and repeating the steps 2) and 3) twice to finish the concave-convex information of six Braille points in one Braille.

FIG. 1 is the sequential numbering of six Braille points, 1, 2 from the left of the first row; the second row is 3 and 4 from the left; the third row is 5, 6 from the left.

FIG. 2 shows the order of touching and reading six Braille points, and the order of sliding the fingers left and right is denoted as (a), (b), (c), (d), (e) and (f), respectively.

And continuously sliding the fingers left and right for six times to complete the touch reading of one Braille. After one braille is read, the finger needs to be lifted from the screen, and then the next braille is read by clicking the screen sliding finger again.

When the finger leaves the screen, the currently touched braille state is reset. Clicking the screen again to slide to the left restarts touching the first braille point of the braille. The touch reading state resetting method may provide a function of re-touch reading the current braille. Namely, if the information of the currently touched braille points is uncertain in the touch reading process, the current braille points can be touched and read again by lifting the fingers and clicking and sliding the fingers.

The distance threshold value of sliding of the finger in different directions in the touch reading process is set to be n pixels (n is larger than or equal to 4), namely the gesture of sliding in the direction is judged when the finger continuously slides in the same direction and exceeds the n pixels, and the mobile phone immediately responds to the existence of the vibration touch to feed back to the user. Therefore, the user can slide the finger for a short distance to sense the touch feeling and quickly slide the finger back and forth left and right to read the braille.

Fig. 4 illustrates a method for reading a braille character r by touch. The information of the six Braille points concave-convex of the Braille character r is shown in figure 3, and the 1 st, 3 rd, 4 th and 5 th Braille points are convex; the 2 nd and 6 th braille points are concave. First, clicking the screen to slide the screen leftwards to read the concave-convex information of the first braille point by the finger, and if the first braille point of the braille character r is convex, providing vibration feedback to the touch screen through the vibration module, as shown in fig. 4 (a). After the concave-convex information of the first braille point is sensed, the finger is slid rightwards to sense the concave-convex information of the second braille point, at the moment, the corresponding second braille point is concave, and vibration feedback is not provided for the touch screen through the vibration module, as shown in fig. 4 (b). And then, sliding the finger to read the information of the third braille point to the left, wherein the corresponding third braille point is convex, and providing vibration feedback to the touch screen through the vibration module, as shown in fig. 4 (c). And sequentially sliding the finger to read the information of the fourth braille point to the right, wherein the corresponding fourth braille point is convex, and providing vibration feedback to the touch screen through the vibration module, as shown in fig. 4 (d). And then, sliding the finger to read the information of the fifth braille point to the left, wherein the corresponding fifth braille point is convex, and providing vibration feedback to the touch screen through the vibration module, as shown in fig. 4 (e). And finally, sliding the finger to the right to read the information of the sixth braille point, wherein the corresponding sixth braille point is concave, and the vibration module does not provide vibration feedback for the touch screen, as shown in fig. 4 (f).

According to the method, the actual test of 8 users shows that the speed of touch reading can reach 1.5 seconds at the highest speed, the method can be used proficiently through 2-minute practice, the method is improved by 3 times compared with the original reading time of 4-5 seconds, the efficiency is high, the blind touch reading mode is changed, and the touch reading efficiency of blind users is improved.

Claims

1. A left-right sliding gesture braille touch reading system is characterized in that:

the Braille conversion module converts the character information into six-point Braille, converts the concave-convex information of the six-point Braille into Braille signals and sends the Braille signals to the control module;

the finger sliding direction is divided into a left direction and a right direction along the surface of the touch screen, and the control module judges whether the finger sliding direction is the left direction or the right direction according to the finger touch position and the change information of the finger touch position in time sequence;

the device comprises a vibration module, a touch screen and a touch screen, wherein the vibration module is connected to the touch screen or directly connected to a finger and feeds back the finger through vibration;

the control module generates a vibration control signal according to the sliding direction of the fingers and the information of the concave-convex surface of the Braille point currently presented in the Braille signals, and sends the vibration control signal to the vibration module to enable the vibration module to vibrate the fingers and feed back the concave-convex touch feeling of the Braille corresponding to the touch screen;

the control module controls the touch reading process in the following way:

the method comprises the following steps that a finger slides on a touch screen in a one-way mode to represent touch reading of one Braille point in six Braille points, the touch screen vibrates to correspond to the Braille point representing touch reading to be a convex surface, and the touch screen does not vibrate to correspond to the Braille point representing touch reading to be a concave surface;

the process of touching and reading the braille is as follows: the method comprises the steps that a sliding finger is touched from one side edge to the other side edge of a touch screen, concave-convex information of one Braille point in six Braille points is touched and read through the touch screen according to vibration feedback, and all six Braille points in one six Braille point are touched and read by repeating unidirectional sliding operation for six times;

in the process of touch reading one Braille, the presenting sequence of six Braille points is from top to bottom according to rows, and the touch reading is performed from left to right in each row, namely, the touch reading is performed sequentially from the left of the first row to the left of each row from top to bottom;

and in the process of repeating the unidirectional sliding operation for six times, the sliding direction of the next time is opposite to the sliding direction of the last time.

2. A left-right slide gesture Braille touch-reading system according to claim 1, wherein: the touch screen, the Braille conversion module, the control module and the vibration module are all integrated in the integrated equipment.

3. A braille touch reading method based on left-right sliding gesture touch is characterized in that:

the method comprises the following steps that a finger slides on a touch screen, the touch screen provides vibration according to presented Braille information, and Braille is touched and read by sensing vibration through sliding of the finger on the whole touch screen to obtain feedback;

4. A left-right sliding gesture Braille touch reading method according to claim 3, characterized in that: the vibration is provided in an intermittent manner.