KR20120010283A - Touch screen mobile interface for the visually impaired and the mobile operating method therefor - Google Patents

Abstract

PURPOSE: A touch screen mobile interface for the blind and a mobile driving method thereof are provided to enable the blind to use a mobile device with touch screen operation by providing icons and menu on a GUI environment. CONSTITUTION: An icon reassigning module(110) reassigns icons and a menu to the position of a number key, a symbol key, and a function key on a dialing screen. A function key extracting module(130) provides data to a controller with the reassigned function key input, the scroll input and a touch on a touch screen. An icon name extracting module(150) extracts a name of the reassigned icons and menu. Text To Speech(TTS) converting module(160) outputs the data received from a text and the icon name extracting module in a voice data.

Description

Touch screen mobile interface for the visually impaired and mobile driving method therefor {Touch screen mobile interface for the visually impaired and the mobile operating method therefor}

The present invention relates to a touch screen-type mobile interface for a visually impaired and a mobile driving method therefor, wherein the visually impaired can use a mobile device provided with an icon and a menu in a graphical user interface (GUI) environment and operated by a touch screen. It is to make it possible. The present invention relocates the graphical user interface (GUI) environment for a dialing screen of a touch screen mobile, adds a text to speech (TTS) function, guides by voice, and performs a function of a function key. The present invention provides an interface that enables the visually impaired to use a touch screen mobile and a driving method therefor.

Background Art In order to overcome the visual obstacles, the touch screen type mobile interface for the visually impaired and the mobile driving method for the visually impaired are known to have a separate mobile device or a separate keypad. have.

As a background art of the present invention, there is a keypad technology of the portable terminal for the visually impaired of the Republic of Korea Patent Publication No. 10-2004-0033845 shown in FIG. This technique is characterized in that, in a keypad of a portable terminal for the visually impaired, a braille indicating a number assigned to each is implemented on the numeric key buttons of the keypad; The braille on each numeric key button is characterized in that the keypad of the portable terminal for the visually impaired is implemented by reading only the braille patterns arranged differently between the numbers in the international standard braille used for the corresponding number. However, this technology has a limitation that cannot be applied to a general touch screen mobile device without a keypad.

As another background art of the present invention, there is a portable telephone technology for the visually impaired (blind person) in Japanese Unexamined Patent Publication No. 2008-252703 in Fig. 2. This technique arranges one of the four parts of a portable telephone into a shape (deformation part) different from another part, and arranges a main feature button in the four parts, and receives a sign language (call). A key point is to arrange | position a button at the said release corner part. In addition, the visually impaired person does not configure unnecessary functions, buttons, or display units, and is limited to functions or devices that are necessary for the visually impaired. Each feature button is displayed with braille, and each button is larger in size than a conventional portable telephone. This technology cannot be applied to a general touch screen type mobile device without a keypad because it must be configured with a separate cell phone and have a button separated by a tactile touch.

As another background technology with respect to the present invention, there is a mobile station user interface and method technology for the use of the visually impaired of the US patent publication US 06675028 shown in FIG. This technique aims at a user interface for a mobile base station that can operate in a wireless communication system as a general cellular communication system. This technique is characterized in that in order to facilitate the use of the user interface and visually impaired user, data received at the mobile base station is displayed in a scrollable braille pattern across the braille keypad at the desired scrolling rate. This technology is a dedicated terminal that provides only received data in Braille, so users cannot make calls or use other additional functions, and they cannot be applied to general touch screen type mobile devices without a keypad.

(Patent Document 1) KR10-2004-0033845 A

(Patent Document 1) JP2008-252703 A

(Patent Document 2) US06675028 B1

An object of the present invention is to provide a touch screen type mobile interface for the visually impaired, which is mounted in a clip on method on a general touch screen type mobile device without a keypad.

In addition, the present invention relocates the graphical user interface (GUI) environment for the dialing screen of the touch screen mobile, adds a text to speech (TTS) function to guide by voice, and also performs a function of a function key. Therefore, to solve the problem to provide an interface and a driving method for enabling the visually impaired to use a touch screen mobile.

The touch screen mobile interface for the visually impaired of the present invention and the mobile driving method therefor,

For dialing screens provided by mobile number keys 0-9 and '*', '#' keys and function keys, the number keys 0-9 and '*', '#' keys and function keys The present invention provides a touch screen mobile interface for the visually impaired, which is equipped with a keypad adapter and a mobile adapter that are matched to a location and mounted in a clip-on manner to the mobile device.

In addition, a touch screen mobile interface for the visually impaired and the mobile with an icon relocation module, text extraction module, function key extraction module, page scroll module, icon name extraction module, TTS (text to speech) conversion module And it provides a mobile driving method for solving the problem.

According to the touch screen-type mobile interface for the visually impaired and the mobile driving method therefor, the touch screen for the visually impaired is clip-on mounted on a general touch screen-type mobile device without a keypad. To provide a mobile interface.

In addition, the present invention relocates the graphical user interface (GUI) environment for the dialing screen of the touch screen mobile, adds a text to speech (TTS) function to guide by voice, and also performs a function of a function key. Therefore, there is a technical effect of providing an icon and a menu to a graphical user interface (GUI) environment and allowing a visually-impaired person to use a mobile device operated by a touch screen.

1 is a background technology of the keypad technology of the portable terminal for the visually impaired
2 is another configuration of the portable telephone for the visually impaired (blind).
3 is another background technology, the configuration of mobile station user interface and method technology for the use of the visually impaired
Figure 4 is a technical idea of the present invention
5 is a keypad adapter according to a first embodiment of the present invention.
Figure 6 is a state in which the keypad adapter and the mobile according to the first embodiment of the present invention is coupled
7 is a detailed structure of a keypad adapter according to a first embodiment of the present invention.
8 shows an embodiment of conductive keypad marking according to a first embodiment of the present invention.
9 is a detailed structure of a mobile adapter according to a second embodiment of the present invention.
10 is a state in which a mobile adapter and a mobile are coupled according to a second embodiment of the present invention.
11 shows an embodiment of marking for recognizing an opening 230b in a second embodiment of the present invention.
Figure 12 is an example of a configuration to perform the icon or menu selection function by mounting the keypad adapter or mobile adapter of the present invention to the mobile
Figure 13 is an example of a configuration to perform a phone call application by mounting the keypad adapter or mobile adapter of the present invention to the mobile
Figure 14 is a configuration of the mobile of the present invention
15 is a flowchart illustrating the operation of an icon menu screen of the present invention.
16 is a flowchart illustrating the operation of the application menu screen of the present invention.

The following merely illustrates the principles of the invention. Accordingly, those skilled in the art will be able to devise various apparatuses which, although not explicitly described or shown herein, embody the principles of the invention and are included in the concept and scope of the invention. Furthermore, all of the conditional terms and embodiments listed herein are, in principle, intended only for the purpose of enabling understanding of the concepts of the present invention, and are not intended to be limiting in any way to the specifically listed embodiments and conditions .

The above objects, features and advantages will become more apparent from the following detailed description in conjunction with the accompanying drawings. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

Figure 4 shows the technical idea of the present invention. The touch screen type mobile interface for the visually impaired of the present invention and the mobile driving method therefor are provided with a function in a graphical user interface (GUI) environment as shown in FIG. In order to enable the visually impaired to use the mobile device operated by the. As illustrated in (b) of FIG. 2, the touch screen mobile 100 may provide a dialing screen provided with number keys 0 to 9 and keys and function keys of '*' and '#'. The present invention provides a voice guide by relocating a graphical user interface (GUI) environment and adding a text to speech (TTS) function to the dialing screen of the touch screen mobile 100 as shown in FIG. In addition, by providing a function of the function key (function key) also provides an interface and a driving method for the visually impaired can use the mobile screen of the touch screen method.

5 shows a keypad adapter for a touch screen type mobile interface according to a first embodiment of the present invention. Keypad adapter 200a of the present invention is provided on the basis of the dialing screen provided by the number keys and '*', '#' keys and function keys of 0 to 9 of the mobile 100, the 0 to 9 It is configured to be detachable in a clip-on (clip on) method to the mobile 100 by having a keypad that is matched to the number keys and '*', '#' keys and function key positions.

Figure 6 shows a state in which the keypad adapter and the mobile according to the first embodiment of the present invention is coupled. As described above, the keypad adapter 200a according to the first embodiment of the present invention is provided based on the dialing screen of the mobile 100, and the number keys 0 to 9 and keys of '*' and '#'. And it is mounted in a clip-on (clip on) method to the mobile 100 to match the position of the function key respectively. Although not shown in the drawing, the keypad adapter 200a may further include an additional fixing means that fits the form factor of the mobile 100 so that the keypad adapter 200a can be more accurately mounted at the key position of the mobile 100. have.

7 shows a detailed structure of a keypad adapter according to the first embodiment of the present invention. A cross-sectional view of A-A 'is shown at the bottom of the keypad adapter 200a of the present invention shown at the top of the figure, and the conductive keypad 230a is provided on the surface corresponding to the touch screen of the mobile 100. ) Is provided, and the touch screen side coupling portion of the conductive keypad 230a is provided with a projection 220 made of an insulating material to maintain insulation between the conductive keypad 230a and the touch screen without pressing the conductive keypad 230a. do. The conductive keypad 230a may be made of a conductive rubber or a conductive polymer polymer having elasticity, and the user may contact the touch screen corresponding to the conductive keypad 230a by pressing the conductive keypad 230a.

The conductive keypad 230a may be integrally formed with all of the keys as a conductive material. If necessary, the conductive keypad 230a may be electrically insulated for each key, and may be made of conductive rubber or a conductive polymer polymer only between the conductive keypad 230a and the touch screen. It may be configured to include.

At this time, the keypad adapter (200a), the number of the number key from 0 to 9 and '*', '#' key and the function key position of the dialing screen to use the touch screen function of the mobile 100 to be described later Except the remaining touch screen portion is configured.

8 shows embodiments of marking conductive keypad 230a in a keypad adapter according to a first embodiment of the present invention. The conductive keypad 230a as a touch screen type mobile interface for the visually impaired of the present invention constitutes a marking in a concave-convex manner in order to recognize by touch on an upper surface thereof. (A) of the figure is an example in which the letter display of the key is provided with the numbers 0 to 9 in Braille, the notation of '*', '#' and function keys, respectively. (B) of the figure is an example of marking only the position of '5' of the number key so that the surrounding keys can be manipulated around it. As shown in (c) and (d), keys are touched. It is also possible to mark distinguishable symbols, where the setting of the symbols can be adopted as necessary for easier identification.

9 shows a detailed structure of a mobile adapter according to a second embodiment of the present invention. The cross-section of B-B 'is shown at the bottom for the mobile adapter 200b of the present invention shown at the top of the figure, and the opening 230b is provided on the surface of the mobile adapter 200b corresponding to the touch screen of the mobile 100. Is provided. The mobile adapter 200b is made of an insulating material, and the user may make a contact operation on the touch screen through the opening 230b.

At this time, the mobile adapter 200b is similar to the keypad adapter 200a described above, in order to use the touch screen function of the mobile 100 which will be described later. Touch screen portion except for the position of the key and the function key of '.

FIG. 10 illustrates a state in which a mobile adapter and a mobile are coupled according to the second embodiment of the present invention. As described above, the mobile adapter 200b according to the second embodiment of the present invention is provided on the basis of the dialing screen of the mobile 100, and the number keys up to 0-9 and keys of '*' and '#'. And a clip on method in the mobile 100 so that the openings 230b are respectively matched to the function key positions. Although not shown in the drawing, the mobile adapter 200b may further include an additional fixing means that fits the form factor of the mobile 100 so that the mobile adapter 200b can be more accurately mounted at the key position of the mobile 100. have.

11 illustrates embodiments of marking for recognizing an opening 230b in a mobile adapter according to a second embodiment of the present invention. The mobile adapter 200b as a touch screen type mobile interface for the visually impaired of the present invention configures the marking in a concave-convex manner in order to recognize the touched part in the peripheral frame portion of the opening 230b. (A) of the figure is an example provided with the letter display of the keys, respectively, with numbers from 0 to 9 in Braille and the display of '*', '#' and function keys. (B) of the figure is an example of marking only the position of '5' of the number key so that the surrounding keys can be manipulated around it. As shown in (c) and (d), keys are touched. It is also possible to mark distinguishable symbols, where the setting of the symbols can be adopted as necessary for easier identification.

FIG. 12 illustrates an example of a configuration in which a keypad adapter or a mobile adapter of the present invention is mounted on a mobile device to perform an icon or menu selection function of a graphical user interface (GUI) environment. The touch screen-based mobile interface for the visually impaired and the mobile driving method therefor relocate a graphical user interface (GUI) environment of the mobile 100. The icons or menus of the graphical user interface (GUI) environment are provided on the basis of the dialing screen of the mobile 100, and are rearranged in the positions of the number keys 0 to 9 and the keys and function keys of '*' and '#'. do. FIG. 12A illustrates a function key for relocating an icon or menu to a number key of 0 to 9 for a dialing screen of the mobile 100 and switching to a previous screen, a function key for switching to a previous screen, and voice conversion. Shows an example in which the function key 240 is assigned. At this time, up to 12 icons or menus may be arranged on one screen, but may be rearranged to a smaller number for easier reading.

In addition, FIG. 12B illustrates mounting of the keypad adapter 200a or the mobile adapter 200b of the present invention to the mobile 100 in a clip on manner, and driving of the remaining exposed touch screen parts. do. In the present invention, the exposed touch screen other than the keypad adapter 200a or the mobile adapter 200b portion is configured to perform a function of a function key. As an example, first, the touch screen is used as a scroll function. The scroll function is provided as a basic function of the mobile 100, but in the present invention, to provide a continuous scroll function by moving while maintaining a contact on the touch screen, the scroll function is to be scrolled in units of pages. Shall be. Secondly, the present invention also has a voice output function according to the touch of the touch screen. For example, when the touch screen is touched, the name, menu, and text information of an icon on the screen is output as a voice.

FIG. 13 shows an example of a configuration in which a keypad adapter or a mobile adapter of the present invention is mounted on a mobile to perform a telephony application in a graphical user interface (GUI) environment. The icons of the graphical user interface (GUI) environment are provided based on the dialing screen of the mobile 100, and the number keys of 0 to 9, the keys of '*' and '#' and the function keys 240 are disposed. . FIG. 12A illustrates an example of rearranging icons on the number keys 0 to 9, the '*' key, the '#' key, and the function key 240 on the dialing screen of the mobile 100.

Also, FIG. 13B shows driving of the remaining exposed touch screen portion in the telephony application of the present invention. In the telephony application of the present invention, an exposed touch screen other than the keypad adapter 200a or the mobile adapter 200b portion is configured to perform a function of a function key. For example, the touch screen is also used as a call key function. The call key function is provided as a basic function of the mobile 100, but in the present invention, when the touch screen is touched once in the phone call application, the call function is performed. Return to the previous screen, a specific configuration for this will be described later.

14 is a diagram illustrating a configuration of a mobile for a touch screen type mobile interface for a visually impaired and a mobile driving method therefor. Hereinafter, a description will be given of the mobile 100 of FIG. 14 by dividing the existing components and the characteristic components of the present invention.

Mobile 100 of the present invention is a communication component 101, the display 102, the touch screen controller 103, the clock timer 104, D / A conversion as the existing components shown in the upper portion separated by a dotted line in the figure A unit 105, an A / D converter 106, a controller 107, and an application module 108 are provided.

The communication unit 101 performs transmission and reception of voice and data of the mobile under the control of the controller 107, and the display 102 displays information and data provided from the controller 107. In the present invention, the display 102 may not directly display information and data provided from the controller 107, and in this case, the controller 107 stores the displayed data in a separate storage location. The touch screen controller 103 detects a signal input from a touch screen sensor provided to overlap the display 102 and provides it to the controller 107. The clock timer 104 sets a time according to the control of the controller 107 or performs a function of a timer, and the controller 107 measures the time until the next touch after touching the touch screen, based on a predetermined time. It is determined whether it is touched once or twice. The D / A converter 105 converts voice or audio information received by the communication unit 101 and provides the external earphone jack 160 through a speaker or an audio jack built in the mobile 100. The A / D converter 106 converts a signal provided through a microphone or an external microphone jack 170 embedded in the mobile 100 into digital data and provides the converted data to the controller 107, and the controller 107 is an application module. According to the execution of 108, data provided from the A / D conversion unit 106 is transmitted to the communication unit 101 or internal processing is performed. The application module 108 performs an application function using the mobile 100 through the controller 107 and stores a plurality of functions and is selected by a user, and the touch screen method for the visually impaired of the present invention. The application of the mobile interface and the method for driving the mobile for the same is provided in the application module (108).

Mobile 100 of the present invention is an icon rearrangement module 110, text extraction module 120, function key extraction module 130, page scroll module as a characteristic component of the present invention shown in the lower portion separated by a dotted line of the drawing 140, an icon name extraction module 150, and a text to speech (TTS) conversion module 160.

Icon relocation module (110)

  The controller 107 performs an application module 108 to display

  Icons and menus displayed on the 102 may be transferred through the icon relocation module 110.

  Relocate.

  In this case it is not necessary to drive the display 102 and the controller 107

  Icon relocation module 110 through the icon or menu of the mobile 100

  Based on the dialing screen, numbers from 0 to 9, keys of '*' and '#' and function keys

  Repositioning to the position and corresponding detection area of the touch screen controller 103

  Drive to match the dialing screen.

Text Extraction Module 120

  The text extraction module 120 is provided to the display 102 by the controller 107.

  Extracts text data. The extracted text data is a TTS conversion module.

  The module 160 outputs the voice data through the D / A converter 105.

Function key extraction module 130

  Re-icon the icons and menus that the controller 107 shows on the display 102.

  Input of '*', '#' keys and function keys rearranged through the batch module 110,

  Scroll input of touch screen, one touch and two touch touch screen

  Input through the control unit 103 and input of '*' and '#' keys and function keys;

  The controller 107 provides information of the function key input to the touch screen.

-Page scroll module 140

  As described above, the controller 107 performs the application module 108

  The icon relocation module 110 may be assigned to icons and menus that are displayed on the display 102.

  To relocate. At this time, the page scroll module 140 is an application

  One screen with icons and menus to display on display 102 from module 108

  The controller 107 in units of one page,

  When the controller 107 is provided with a scroll input from the function key extraction module 130

  Icons and menus scrolled by page through page scroll module 140

  New is relocated via the icon relocation module 110.

Icon Name Extraction Module 150

  Controller 107 performs application module 108 to display 102.

  When the icons and menus to be displayed are rearranged through the icon relocation module 110,

  There are only GUI icons and menus on the screen. Icon Name Additional

  The exit module 150 may be configured to execute an application for the rearranged icons and menus.

  Refer to module 108 to extract the names of icons and menus placed on the screen,

  To the controller 107.

Text to speech (TTS) conversion module 160

  The controller 107 extracts the text extraction module 120 and the icon name extraction module 150.

  Data from the data to speech conversion module (TST) 160.

  If so, the text to speech (TTS) conversion module 160 converts the data into speech data.

Output through the conversion unit 105.

15 is a flowchart illustrating a method of driving an icon menu screen as a mobile driving method for a touch screen type mobile interface for the visually impaired according to the present invention. If this is described in the order of the working relationship as follows.

a) When the icon screen is started, the controller 107 of the mobile 100 controls the display 102 and the touch screen controller 103 to number keys 0 to 9 and '*' based on the dialing screen of the mobile 100. Switch to keypad mode consisting of '#' key and function keys. (S100)

b) Then, the controller 107 uses the icon rearrangement module 110 to display icons or menus with number keys of 0 to 9 and keys of '*' and '#' and function keys based on the dialing screen of the mobile 100. The mobile device 100 is rearranged to the position of the touch screen controller 103 so that the sensing area of the touch screen controller 103 corresponds to the keys of the dialing screen.

c) Next, the controller 107 determines whether the function key of the touch screen has been touched from the touch screen controller 103 (S120).

d) If the function key is not touched in the step S120, the controller 107 determines whether the rearranged icon is touched (S130).

e) If it is determined in step S130 that the rearranged icon is touched, the controller 107 extracts the name of the touched icon through the icon name extraction module 150. (S140)

f) Then, the controller 107 provides the name of the extracted icon to the TTS conversion module 160 and outputs it as voice data through the D / A conversion unit 105 (S150).

g) Next, the controller 107 determines whether the same icon is touched again within the time set by the output timer 104 (S160).

h) If it is determined that the same icon is touched again within the time set in the step S150, the controller 107 calls and executes an application corresponding to the icon (S170).

i) When the function key is touched in the step S120, the controller 107 determines whether the touched function key is a scroll function key.

j) If the function key touched in step S200 is not a scroll function key, the controller 107 determines whether the current screen is an icon screen.

k) If the current screen is an icon screen in step S210, the controller 107 extracts all the names of the icons of the screen through the icon name extraction module 150. (S220)

l) Then, the controller 107 provides the name of the icon of the extracted screen to the TTS conversion module 160 and outputs it as voice data through the D / A converter 105 (S230), and then the controller 107. ) Is performed again from step S120 of determining whether a function key of the touch screen has been touched by the touch screen controller 103.

m) If the current screen is not an icon screen in step 210, the controller 107 extracts both the current screen name and the text on the screen through the text extraction module 120 (S250).

n) Then, the controller 107 provides the extracted current screen name and the text on the screen to the TTS conversion module 160, and outputs them as voice data through the D / A converter 105 (S230). Operation 107 is performed again from the step S120 of determining whether a function key of the touch screen has been touched by the touch screen controller 103.

o) If the function key touched in the step (S200) is a scroll function key, the controller 107 scrolls the page of the screen through the page scroll module 140 (S240), and the icon or the icon through the icon rearrangement module 110 The menus are rearranged to the positions of the number keys 0 to 9 and the positions of the keys and function keys of '*' and '#' based on the dialing screen of the mobile 100, and the sensing area of the touch screen controller 103 corresponding thereto is The operation is repeated again from the step S110 of driving the mobile 100 to match the keys of the dialing screen.

p) when the icon relocated in step S130 is not touched; When the same icon is not touched again within the time set by the output timer 104 in step S160; When the execution is completed by calling the application corresponding to the icon in step S170; the controller 107 determines whether the function key of the touch screen has been touched by the touch screen controller 103 (S120). To perform.

16 is a flowchart illustrating a method of driving an application menu screen as a mobile driving method for a touch screen type mobile interface for the visually impaired according to the present invention. If this is described in the order of the working relationship as follows.

a ') When the application screen is started, the controller 107 of the mobile 100 sets the display 102 and the touch screen controller 103 to a number key of 0 to 9 and' * 'based on the dialing screen of the mobile 100. Switch to keypad mode consisting of ',' # 'key and function keys. (S300)

b ') Then, the controller 107 uses the icon rearrangement module 110 to display the application menu key names with the number keys 0 to 9 and the keys' *', '#', and the like based on the dialing screen of the mobile 100. The mobile device 100 is rearranged to the position of the function key and the mobile device 100 is driven so that the sensing area of the touch screen controller 103 corresponds to the keys of the dialing screen (S310).

c ') Next, the controller 107 determines whether the function key of the touch screen has been touched from the touch screen controller 103 (S320).

d ') If the function key is not touched in the step S320, the controller 107 determines whether the rearranged application menu key is touched (S330).

e ') If it is determined in step S330 that the application menu key is touched, the controller 107 extracts the name of the touched application menu key through the icon name extraction module 150 (S340).

f ') Then, the controller 107 stores the name of the extracted application menu key (S350).

g ') Next, the controller 107 provides the name of the extracted application menu key to the TTS conversion module 160 and outputs it as voice data through the D / A conversion unit 105 (S360).

h ') Following step S360, the controller 107 determines whether the same application menu key is touched again within the time set by the output timer 104 (S370).

i ') If it is determined that the same icon is touched again within the time set in the step S370, the controller 107 links the application menu key input information to the application (S380).

j ') If the function key is touched in the step S320, the controller 107 determines whether the touched function key is a scroll function key.

k ') If the function key touched in step S400 is not a scroll function key, the controller 107 extracts the text on the screen through the text extraction module 120 (S420).

l ') Then, the controller 107 provides the extracted text on the screen to the TTS conversion module 160 and outputs it as voice data through the D / A converter 105 (S430), and then the controller 107. Extracts whether the function key touched by the touch screen controller 103 is touched once or twice within the time set by the output timer 104 (S440) and links the number of times to the application (S380).

m ') If the function key touched in the step (S400) is a scroll function key, the controller 107 scrolls the page of the screen through the page scroll module 140 (S240) and the application through the icon rearrangement module 110. Menu key names are rearranged to the positions of the number keys 0 to 9 and the positions of the keys and function keys of '*' and '#' based on the dialing screen of the mobile 100 and the corresponding detection of the touch screen controller 103. Driving the mobile 100 so that the area matches the keys of the dialing screen (S310);

n ') when it is determined in step S330 that the application menu key is not touched; When it is determined that the same icon is not touched again within the time set in the step S370; If the application menu key input information is linked to the application (S380), the controller 107 determines whether the function key of the touch screen has been touched by the touch screen controller 103 (S320). .

As described above, the touch screen mobile interface for the visually impaired and the mobile driving method therefor relocate a graphical user interface (GUI) environment to a dialing screen of a touch screen mobile and provide text to speech (TTS). By adding a function to guide the voice and functioning as a function key, a menu and an icon are provided by providing an interface that enables the visually impaired to use a touch screen mobile and a driving method therefor. Allows the visually impaired to use a mobile device operated by a touch screen.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. And various modifications and variations are possible within the scope of the appended claims.

100: mobile 101: communication unit
102 display 103 touch screen control
104: clock timer 105: D / A converter
106: A / D converter 107: controller
108: application module 110: icon relocation module
120: text extraction module 130: function key extraction module
140: page scroll module 150: icon name extraction module
160: TTS (text to speech) module 200a: keypad adapter
200b: mobile adapter 220: protrusion
230a: conductive keypad 230b: opening
240: function keys

Claims (21)

In the touch screen mobile interface for the visually impaired,
For the dialing screens provided by the number keys up to 0-9 and '*', '#' keys, and function keys of the mobile 100, the number keys up to 0-9 and '*', '#' keys Touch screen type mobile interface for the visually impaired, characterized in that it has a keypad adapter that is matched to the position of the function key and is mounted on the mobile clip on (clip on) method
The keypad adapter of claim 1, wherein
A conductive keypad is provided on a surface of the keypad adapter corresponding to the touch screen of the mobile device.
The touch screen side coupling portion of the conductive keypad is provided with a projection of an insulating material, and is configured to maintain insulation between the conductive keypad and the touch screen without pressing the conductive keypad.
The method of claim 2, wherein the conductive keypad,
A touch screen type mobile interface for the visually impaired, which is made of a conductive rubber or a conductive polymer polymer having elasticity, and the user presses the conductive keypad so that a contact operation is made on the touch screen corresponding to the conductive keypad.
The method of claim 2, wherein the conductive keypad,
Touch screen type mobile interface for the visually impaired, characterized in that all the keys are integrally made of conductive material.
The method of claim 2, wherein the conductive keypad,
Touch screen type mobile interface for the visually impaired, characterized in that each key is electrically insulated and configured to have a conductive rubber or a conductive polymer polymer material only between the conductive keypad and the touch screen.
The keypad adapter of claim 1, wherein
Touch screen type mobile for the visually impaired, characterized in that the part of the touch screen except for the position of the number keys of 0-9 and '*', '#' keys and function keys of the dialing screen of the mobile is exposed. interface
The method of claim 2, wherein the conductive keypad,
In order to recognize by touch on the upper surface, the marking is constructed in an uneven manner,
The marking has a number 0 to 9 in Braille and '*', '#', and the display of the function keys, or
Mark only the '5' position of the number key so that you can operate the surrounding keys around it,
Touch screen type mobile interface for the visually impaired, characterized by marking a symbol that can distinguish each key by tactile sense
In the touch screen mobile interface for the visually impaired,
For dialing screens provided by mobile number keys 0-9 and '*', '#' keys and function keys, the number keys up to 0-9 and '*', '#' keys and function keys Touch screen type mobile interface for the visually impaired, characterized in that it has a mobile adapter that is matched to the position of the clip mounted on the mobile (clip on) method
The method of claim 8, wherein the mobile adapter,
It is made of insulating material,
For the dialing screens provided by the mobile number keys 0 to 9 and the '*', '#' keys and function keys, the number keys up to 0-9 and the '*' and '#' keys and function keys It has an opening that is matched to the position of
Touch screen mobile interface for the visually impaired, characterized in that it has a mobile adapter that is mounted on the mobile clip on (clip on) method
The mobile adapter of claim 9, wherein
In order to recognize by the tactile sensation to the peripheral frame part of the opening, the marking is constructed in an uneven manner.
The marking has a number between 0 and 9 in Braille and '*', '#', and a function key,
Mark only the '5' frame of the number key so that the surrounding keys can be operated around it.
Touch screen type mobile interface for the visually impaired, characterized by marking a symbol that can distinguish each key by tactile sense
The mobile adapter of claim 8,
Touch screen type mobile for the visually impaired, characterized in that the part of the touch screen except for the number keys of '0' to '9', '*', '#' keys, and function keys of the dialing screen of the mobile is exposed. interface
In the mobile equipped with a communication unit, a display, a touch screen controller, a clock timer, a D / A converter, an A / D converter, a controller, and an application module in a touch screen mobile interface for the visually impaired,

An icon rearrangement module for rearranging icons and menus displayed on a display by a controller to positions of number keys 0 to 9 and positions of '*' and '#' keys and function keys based on the dialing screen of the mobile;
A text extraction module for extracting text data provided to the display by the controller; and
Input of the function keys rearranged through the icon relocation module, scroll input of the touch screen, single touch and double touch are provided through the touch screen controller, input of the function keys, and information of the function keys input to the touch screen. A function key extraction module for providing a controller to the controller;
A page scroll module that provides icons and menus rearranged and displayed on an display from an application module to the controller in one screen unit, that is, one page unit;
An icon name extraction module for extracting names of icons and menus disposed on the screen with reference to the application module with respect to the rearranged icons and menus and providing them to the controller;
A text-to-speech (TTS) conversion module for outputting data provided from the text extraction module and the icon name extraction module to the voice data through the D / A converter;
Touch screen mobile interface for the visually impaired, characterized in that it comprises a
As a method of driving an icon menu screen in a mobile driving method for a touch screen-based mobile interface for the visually impaired,
a) When the icon screen starts, the controller of the mobile switches the display and touch screen controller to the keypad mode consisting of the number keys 0-9 and the keys and function keys of '*' and '#' based on the dialing screen of the mobile. Step (S100);
b) The controller then rearranges the icons or menus through the icon relocation module to the positions of the number keys 0-9 and the keys and function keys of '*' and '#' based on the dialing screen of the mobile. Driving the mobile so that the sensing area of the screen controller matches the keys of the dialing screen (S110);
c) Next, the controller determines whether the function key of the touch screen is touched from the touch screen controller (S120);
d) if the function key is not touched at step S120, the controller determines whether the rearranged icon is touched at step S130;
e) if it is determined that the relocated icon is touched in the step S130, the controller extracts the name of the touched icon through the icon name extraction module (S140);
f) the controller then provides the name of the extracted icon to the TTS conversion module and outputs it as voice data through the D / A conversion unit (S150);
g) next, the controller determines whether the same icon is retouched within the time set by the output timer (S160);
h) if it is determined that the same icon is touched again within the time set in step S150, the controller calls and executes an application corresponding to the icon (S170);
Mobile driving method for a touch screen-type mobile interface for the visually impaired, characterized in that consisting of
In the step (S120) of claim 13,
i) if the function key is touched in step S120, determining whether the function key touched by the controller is a scroll function key (S200);
j) if the touched function key in step S200 is not a scroll function key, the controller determines whether the current screen is an icon screen (S210);
k) if the current screen is an icon screen in step S210, the controller extracts all names of icons of the screen through an icon name extraction module (S220);
l) the controller provides the name of the icon of the extracted screen to the TTS conversion module and outputs it as voice data through the D / A converter (S230); Via
Next, the controller determines whether the function key of the touch screen has been touched by the controller from the touch screen controller (S120). The mobile driving method for the touch screen-type mobile interface for the visually impaired may be performed.
In the step (S210) of claim 14,
m) if the current screen is not an icon screen in step 210, the controller extracts both the current screen name and the text on the screen through the text extraction module (S250);
n) then providing the extracted current screen name and the text on the screen to the TTS conversion module and outputting them as voice data through the D / A converter (S230).
Next, the controller determines whether the function key of the touch screen has been touched by the controller from the touch screen controller (S120). The mobile driving method for the touch screen-type mobile interface for the visually impaired may be performed.
In the step (S200) of claim 14,
If the touched function key is a scroll function key, the controller scrolls a page of the screen through the page scroll module (S240);
As the icon relocation module, icons or menus are relocated to the position of the number keys 0-9 and the keys and function keys of '*' and '#' based on the dialing screen of the mobile. Mobile driving method for a touch screen-type mobile interface for the visually impaired, characterized in that the step of repeating the operation from the step of driving the mobile to match the keys of the dialing screen (S110) again
The method of claim 13,
If the icon rearranged in the step (S130) is not touched;
When the same icon is not touched again within the time set by the output timer in step S160;
When the execution is completed by calling the application corresponding to the icon in the step (S170);
The controller is configured to perform the process again from the step S120 of determining whether a function key of the touch screen has been touched by the touch screen controller.
As a method of driving an application menu screen in a mobile driving method for a touch screen-based mobile interface for the visually impaired,
a ') When the application screen starts, the controller of the mobile enters the keypad mode consisting of the number keys of 0-9,' * ','#'keys and function keys based on the dialing screen of the mobile. Switching step (S300);
b ') The controller then relocates the application menu key names to the number keys 0-9 and the positions of the' * 'and'#'keys and function keys based on the mobile dialing screen. Driving the mobile so that the sensing area of the touch screen controller matches the keys of the dialing screen (S310);
c ') next, the controller determines whether a function key of the touch screen has been touched by the touch screen controller (S320);
d ') if the function key is not touched in the step S320, the controller determines whether the rearranged application menu key is touched (S330);
e ') If it is determined in step S330 that the application menu key is touched, the controller extracts the name of the touched application menu key through the icon name extraction module (S340);
f ') Then, the controller stores the name of the extracted application menu key (S350);
g ') Next, the controller provides the name of the extracted application menu key to the TTS conversion module and outputs it as voice data through the D / A converter (S360);
h ') following the step S360, the controller determines whether the same application menu key is touched again within the time set by the output timer (S370);
i ') if it is determined that the same icon is touched again within the time set in the step S370, the controller links the application menu key input information to the application (S380);
Mobile driving method for a touch screen-type mobile interface for the visually impaired, characterized in that consisting of
The step (S320) of claim 18,
j ') when the function key is touched in the step S320, the controller determines whether the touched function key is a scroll function key (S400);
k ') If the touched function key in step S400 is not a scroll function key, the controller extracts text on the screen through the text extraction module (S420).
l ') Then, the controller provides the extracted text on the screen to the TTS conversion module and outputs it as voice data through the D / A converter (S430).
Next, the controller extracts whether the function key touched from the touch screen controller is touched once or twice within the time set by the output timer (S440).
Linking the number of times to the application (S380); Mobile driving method for a touch screen-type mobile interface for the visually impaired
The method of claim 19,
If the touched function key in step S400 is a scroll function key, the controller scrolls a page of a screen through the page scroll module (S240);
Through the icon relocation module 110, the application menu key names are rearranged to the positions of the number keys 0 to 9, the keys of the '*' and '#' keys and the function keys based on the dialing screen of the mobile, and corresponding touch screens. And driving the mobile so that the sensing area of the control unit matches the keys of the dialing screen (S310). The mobile driving method for the touch screen type mobile interface for the visually impaired may be performed again.
The method of claim 18,
If it is determined in step S330 that the application menu key is not touched;
When it is determined that the same icon is not touched again within the time set in the step S370;
And when the application menu key input information is performed by linking to the application (S380);
The method of driving the mobile screen for the touch screen-type mobile interface for the visually impaired may be performed by the controller from the touch screen controller to determine whether the function key of the touch screen is touched (S320).

Images