KR20220109162A - A mobile terminal for prividing virtual keypads for blind person - Google Patents

Abstract

The present invention relates to a portable terminal, capable of displaying a virtual keypad suitable for a tactile sheet for the visually impaired, which comprises: a touch screen; and a control unit for controlling the touch screen to display a first virtual keypad for controlling a screen and an application, and a second virtual keypad for inputting a letter, a number, and a symbol on the touch screen to which a tactile sheet is attached. The control unit controls the touch screen to display the first and second virtual keypads on an area of the touch screen matching the attached tactile sheet, and controls the touch screen to display one of preset key combinations of the second virtual keypad on the basis of attribute information on an application being executed on the touch screen. Therefore, the visually impaired can easily control an application or an application program.

Description

A portable terminal providing a virtual keypad for the visually impaired {A MOBILE TERMINAL FOR PRIVIDING VIRTUAL KEYPADS FOR BLIND PERSON}

The present invention relates to a portable terminal providing a virtual keypad for the visually impaired, and more particularly, to a portable terminal providing a virtual keypad to a touch screen to which a tactile sheet for the visually impaired is attached.

In general, the visually impaired can obtain information through sound or tactile information, but it is difficult to use the latest electronic devices to which a touch screen used while looking with the eyes is applied. The general public can view and obtain various information using a portable terminal, but it is useless to the visually impaired.

Usually, there is a method of attaching a metal palpation plate to the touch panel, but the user must directly check and attach the palpation plate made of Braille or pictogram formed by pressing the base plate, and by touching it, information can be obtained through the terminal. can However, when the palpation plate and the touch screen are combined, a separate assembly tool is provided and the user must directly assemble it. Therefore, there is a problem in that it is difficult to actually perform these tasks alone.

In addition, even when a terminal with a tactile plate is used, in order to find an application or function that a user wants, it is inconvenient to have to select and check the applications or functions one by one in order.

The present invention was devised in view of the above points, and an object of the present invention is to provide a portable terminal capable of displaying a virtual keypad suitable for the palpation sheet on a touch screen to which the palpation sheet is attached.

Another object of the present invention is to provide a portable terminal capable of selectively displaying a key combination of a virtual keypad suitable for a running application or application program.

Another object of the present invention is to provide a portable terminal capable of displaying a virtual keypad that facilitates selection and control of a desired application or application program.

The technical problems to be achieved in the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those of ordinary skill in the art to which the present invention belongs from the description below. will be able

According to an aspect of the present invention, there is provided a portable terminal comprising: a touch screen; A first virtual keypad for controlling a screen and an application and a second virtual keypad for inputting characters, numbers and symbols comprising a controller for controlling the touch screen to display on the touch screen to which a palpation sheet is attached, the controller controls the touch screen to display the first and second virtual keypads in an area of the touch screen that matches the attached palpation sheet, and the second virtual keypad based on attribute information of an application running on the touch screen Controls the touch screen to display one of the preset key combinations.

The control unit determines positions of the first and second virtual keypads to be displayed on the touch screen and a key combination of the first and second virtual keypads according to pre-stored keypad information.

In addition, the control unit requests the user to touch at least one of the protrusions formed on the palpation sheet using a guide voice or a guide screen, and the attachment position of the palpation sheet and the It is possible to determine the type of the palpation sheet. The control unit may determine the size of the palpation sheet and the combination and position of all the protrusions included in the facilitation sheet according to the position of the protrusion touched by the user.

In addition, the control unit determines the positions of the first and second virtual keypads to be displayed on the touch screen according to the attachment position of the palpation sheet and the type of the palpation sheet, and one of the preset key combinations of the second virtual keypad. When is displayed, the output unit is controlled to output a guide voice explaining the displayed key combination.

Also, when the user's drag touch is sensed on the second virtual keypad, the controller may change a key combination of the second virtual keypad according to the drag touch.

Since the portable terminal according to the present invention displays a virtual keypad suitable for the position and type of the palpation sheet attached to the touch screen, the visually impaired person can use the desired tactile sheet together with the portable terminal.

In addition, by selectively displaying a key combination of a virtual keypad suitable for a running application or application, the visually impaired can easily control the application or application.

1 is a diagram illustrating a portable terminal on which a virtual keypad is displayed according to a first embodiment of the present invention.
FIG. 2 is a diagram illustrating some components of the portable terminal of FIG. 1 .
3 is a view showing a state before attaching the palpation sheet according to the first embodiment of the present invention to the portable terminal.
4 is a view showing a state in which the palpation sheet according to the first embodiment of the present invention is attached to the portable terminal.
5 is a diagram illustrating an example of a screen displayed on the portable terminal of FIG. 1 .
6 is a view showing a facilitation sheet according to a second embodiment of the present invention.
7 to 11 are diagrams illustrating a portable terminal on which virtual keypads are displayed according to a second embodiment of the present invention.
12 is a view showing screens that are sequentially executed when the 'phone' key of FIG. 7 is selected.

Hereinafter, the embodiments disclosed in the present specification will be described in detail with reference to the accompanying drawings, but the same or similar components are assigned the same reference numerals regardless of reference numerals, and redundant description thereof will be omitted. The suffixes "module" and "part" for components used in the following description are given or mixed in consideration of only the ease of writing the specification, and do not have distinct meanings or roles by themselves. In addition, in describing the embodiments disclosed in the present specification, if it is determined that detailed descriptions of related known technologies may obscure the gist of the embodiments disclosed in this specification, the detailed description thereof will be omitted. In addition, the accompanying drawings are only for easy understanding of the embodiments disclosed in this specification, and the technical idea disclosed herein is not limited by the accompanying drawings, and all changes included in the spirit and scope of the present invention , should be understood to include equivalents or substitutes.

Hereinafter, a palpation sheet and a portable terminal using the same according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

No. 1 Example

1 shows a portable terminal 10 on which a virtual keypad is displayed according to a first embodiment of the present invention, wherein a first virtual keypad 113 and a second virtual keypad 115 are placed on a predetermined area of the touch screen 111 . shows the displayed portable terminal 10 .

The portable terminal 10 described in this specification is a mobile phone, a smart phone, a laptop computer, a digital broadcasting terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), a navigation system, a slate PC ( slate PC), tablet PC (tablet PC), ultrabook (ultrabook), wearable device (e.g., watch-type terminal (smartwatch), glass-type terminal (smart glass), HMD (head mounted display)) It can be any one. In addition, the portable terminal 10 of the present invention may be configured as another type of terminal capable of wireless communication.

As shown in FIG. 2 , the portable terminal 10 may include an input unit 110 , a communication unit 120 , an output unit 130 , a memory 140 , a control unit 170 , and the like. The portable terminal 10 may be composed of more or fewer components than those listed herein.

Specifically, the input unit 110 may include a camera or image input unit for inputting an image signal, a microphone or an audio input unit for inputting an audio signal, and a user input unit for receiving information from a user. The user input unit is for receiving information from the user, and when information is input through the user input unit, the controller 170 may control the operation of the portable terminal 10 to correspond to the input information. Such a user input unit is a mechanical input means (or a mechanical key, for example, a button located on the front, rear or side of the portable terminal 10, a dome switch, a jog wheel, a jog switch, etc.) ) and a touch input means. As an example, the touch input means consists of a virtual key, a soft key, or a visual key displayed on the touch screen 111 through software processing, or the touch screen (111) may be formed of a touch key (touch key) disposed in a portion other than. On the other hand, the virtual key or the visual key can be displayed on the touch screen 111 while having various shapes, for example, a graphic, a text, an icon, and a video. ) or a combination thereof.

On the other hand, for convenience of explanation, the act of approaching the object on the touch screen 111 without contacting it so that the object is recognized that it is located on the touch screen is called "proximity touch", and , an action of actually touching an object on the touch screen 111 is called “contact touch”. The position where the object is touched in proximity on the touch screen 111 means a position in which the object vertically corresponds to the touch screen 111 when the object is touched in proximity. The proximity sensor may detect a proximity touch and a proximity touch pattern (eg, a proximity touch distance, a proximity touch direction, a proximity touch speed, a proximity touch time, a proximity touch position, a proximity touch movement state, etc.). On the other hand, the controller 170 processes data (or information) corresponding to the proximity touch operation and the proximity touch pattern sensed through the proximity sensor as described above, and further, displays visual information corresponding to the processed data to the touch screen ( 111) can be output. Furthermore, the controller 170 controls the portable terminal 10 so that different operations or data (or information) are processed according to whether a touch on the same point on the touch screen 111 is a proximity touch or a contact touch. can do.

The touch sensor senses a touch (or touch input) applied to the touch screen (or the display unit) using at least one of various touch methods such as a resistive method, a capacitive method, an infrared method, an ultrasonic method, and a magnetic field method.

As an example, the touch sensor may be configured to convert a change in pressure applied to a specific part of the touch screen 111 or a change in capacitance occurring in a specific part of the touch screen 111 into an electrical input signal. The touch sensor may be configured to detect a position, an area, a pressure at the time of touch, an electrostatic capacitance at the time of touch, etc. where a touch object applying a touch on the touch screen 111 is touched on the touch sensor. Here, the touch object is an object that applies a touch to the touch sensor, and may be, for example, a finger, a touch pen, a stylus pen, or a pointer.

As such, when there is a touch input to the touch sensor, a signal(s) corresponding thereto is sent to the touch controller. The touch controller processes the signal(s) and then transmits the corresponding data to the controller 170 . Accordingly, the control unit 170 can know which area of the display unit 111 has been touched, and the like. Here, the touch controller may be a component separate from the controller 170 , or may be the controller 170 itself.

Meanwhile, the controller 170 may perform different controls or may perform the same control according to the type of the touch object that touches the touch screen 111 . Whether to perform different control or the same control according to the type of the touch object may be determined according to the current operating state of the portable terminal 10 or a running application program.

On the other hand, the above-described touch sensor and proximity sensor are independently or in combination, a short (or tap) touch (short touch), long touch (long touch), multi-touch (multi touch), drag touch on the touch screen 111 such as drag touch, flick touch, pinch-in touch, pinch-out touch, swype touch, hovering touch, etc.; Various types of touch can be sensed.

The communication unit 120 enables wireless communication between the portable terminal 10 and a wireless communication system, between the portable terminal 10 and another portable terminal, or between the portable terminal 10 and a server using a wireless communication network. It may include one or more modules. In addition, the communication unit 120 may include one or more modules for connecting the portable terminal 10 to one or more networks. The communication unit 120 may include at least one of a broadcast reception module, a mobile communication module, a wireless Internet module, a short-range communication module, and a location information module.

The output unit 130 is for generating an output related to visual, auditory or tactile sense, and may include at least one of a display unit, a sound output unit, a haptip module, and an optical output unit. The display unit may implement the touch screen 111 by forming a mutually layered structure with the touch sensor or integrally formed therewith. The touch screen 111 may function as a user input unit providing an input interface between the portable terminal 10 and a user, and may provide an output interface between the portable terminal 10 and a user.

The memory 140 stores data supporting various functions of the portable terminal 10 . The memory 140 may store a plurality of application programs (or applications) driven in the portable terminal 10 , data for operation of the portable terminal 10 , and commands. At least some of these application programs may be downloaded from an external server through wireless communication. In addition, at least some of these application programs may exist on the portable terminal 10 from the time of shipment for basic functions (eg, incoming calls, outgoing functions, message reception, and outgoing functions) of the portable terminal 10 . . Meanwhile, the application program may be stored in the memory 140 , installed on the portable terminal 10 , and driven to perform an operation (or function) of the portable terminal 10 by the controller 170 .

In addition to the operation related to the application program, the controller 170 generally controls the overall operation of the portable terminal 10 . The controller 170 may provide or process appropriate information or functions to the user by processing signals, data, information, etc. input or output through the above-described components or by driving an application program stored in the memory 140 . Also, in order to drive an application program stored in the memory 140 , the controller 170 may operate at least two or more of the components included in the portable terminal 10 in combination with each other.

As shown in FIG. 1 , the first virtual keypad 113 is for controlling a screen, an application, and an application program, and includes a direction key, an OK key, a home key, and the like. The second virtual keypad 115 is for inputting letters, numbers, and symbols, and is located below the first virtual keypad 113 . The second virtual keypad 115 is converted into a Korean mode, a foreign language mode, a number mode, a symbol mode, etc. during use, and a key combination of each mode is preset and stored in the memory 140 .

3 shows a state before the facilitation sheet 30 according to the first embodiment of the present invention is attached to the portable terminal 10, and FIG. 4 shows the facilitation sheet 30 according to the first embodiment of the present invention. The state attached to the portable terminal 10 is shown.

, ＃, ＊ 등의 형상을 가지며 촉지시트(30)의 표면 위로 돌출되도록 형성된다. The palpation sheet 30 according to the first embodiment of the present invention includes a plurality of protrusions that are matched one-to-one with virtual keys included in the first and second virtual keypads 113 and 115 . These projections are ｜, │, ―, ＝,

, #, *, etc. are formed to protrude above the surface of the facilitating sheet 30 .

The controller 170 controls the touch screen 111 to display the first and second virtual keypads 113 and 115 on a predetermined area of the touch screen 111 to which the palpation sheet 30 is attached. In particular, the control unit 170 controls the touch screen to display the virtual keys of the first and second virtual keypads 113 and 115 in the area of the touch screen 111 that matches the positions of the protrusions formed on the attached palpation sheet 30 . 111) is controlled.

At this time, the controller 170 determines a combination of virtual keys to be displayed on the touch screen 111 according to the keypad information stored in the memory 140 or first and second virtual keypads ( The positions 113 and 115 and the positions of the virtual keys may be determined.

Here, the pre-stored keypad information may include information on one or more virtual keypads, for example, data indicating the size of each virtual keypad, data indicating the position to be displayed of the virtual keys included in each virtual keypad, each It includes at least one of data representing a combination/arrangement of virtual keys and data representing a type or ID of a preselected virtual keypad. Accordingly, the controller 170 may display the first and second virtual keypads 113 and 115 pre-selected according to the pre-stored keypad information on a designated area of the touch screen 111 .

The keypad information may be received from another portable terminal or server using the communication unit 120 .

On the other hand, the control unit 170 can automatically recognize the actual position and type of the attached palpation sheet 30. For this purpose, first, the control unit 170 uses at least one of the protrusions using a voice guidance or a guidance screen. Ask the user to touch .

Since the user matches and memorizes the positions of the projections and the positions of the virtual keys, he or she can find and press the desired virtual key by fumbling the projections. The control unit 170 detects the position of the protrusion touched by the user according to the guidance voice or the guidance screen, and according to the detected position, the actual position of the area to which the palpation sheet 30 is attached, and the type of palpation sheet (size of the palpation sheet) , a combination/arrangement of all protrusions, positions of all protrusions, etc.).

For example, the controller 170 outputs a guide voice so as to press the virtual keys corresponding to '1', '2', '3', and 'cancel' in order, and touches the protrusions that the user remembers to create a virtual When the keys are pressed, the control unit 170 compares the positions of the protrusions (virtual keys corresponding to '1', '2', '3', and 'cancel') touched by the user with the stored keypad information, and the corresponding tactile information Determine the attachment position and type of the sheet.

And, the control unit 170 determines the positions of the first and second virtual keypads 113 and 115 to be displayed on the touch screen 111 according to the attachment position of the palpation sheet 30 and the type of the palpation sheet 30, and The touch screen 111 is controlled to display the virtual keys of the 1 and 2 virtual keypads 113 and 115 . To this end, the stored keypad information preferably further includes data indicating the type/ID of the first and second virtual cappads 113 and 115 designated for the type of each palpation sheet 30 . Accordingly, the control unit 170 can select the first and second virtual keypads 113 and 115 suitable for the type of the corresponding palpation sheet 30, and the selected first and second virtual keypads 113 and 115 to match the actual position to which the palpation sheet 30 is attached. 1 and 2 virtual keypads 113 and 115 may be displayed on the touch screen 111 .

In addition, the control unit 170 reads the attribute information of the application or application actually executed on the touch screen 111 from the memory 140, and a preset key of the second virtual keypad 115 based on the attribute information. Controls the touchscreen 111 to display one of the combinations. Here, the attribute information includes information indicating a language used, a usage purpose, a compatible operating system (OS) program, a compatible portable terminal, etc. related to the application and the application program. The preset key combination of the second virtual keypad 115 means that related keys are preset according to the Korean mode, the foreign language mode, the numeric mode, and the symbol mode. may have been set.

When one of the preset key combinations of the second virtual keypad 115 is displayed on the touch screen 111, the control unit 170 controls the output unit 130 to output a guide voice explaining the corresponding key combination. . For example, when the displayed key combination is the Korean mode, the control unit 170 outputs a guide voice of 'This is a Korean mode key combination'.

On the other hand, the user can change the key combination of the second virtual keypad 115 displayed on the touch screen 111. When the user drags and touches the second virtual keypad 115 in a certain direction, the control unit 170 It is possible to detect and change the key combination of the second virtual keypad 115 according to the direction of the drag touch. For example, when the user horizontally drags and touches from right to left, the controller 170 may change the key combination in the Korean mode to the key combination in the foreign language mode.

When the virtual keypads 113 and 115 are displayed according to the above methods, the user can select and use the desired virtual keypad even if the portable terminal 10 of a different model or specification is changed. In particular, the visually impaired can continue to use the virtual keypad familiar to them by using the tactile sheet 30 .

When the first and second virtual keypads 113 and 115 are displayed on the touch screen 111, the user can select a protrusion of the tactile sheet 30 and press a desired virtual key, and use a preset shortcut key to select a desired application or function. can be easily accessed. Conventionally, in order to access the desired content, it is necessary to check the desired content one by one while listening to the voice output from the portable terminal 10 . However, since you can press a shortcut key using the projection, it is possible to run the application intuitively and quickly. Stopping or terminating the application can also be executed with a shortcut key. For example, when the 'home key' in the upper center of the first virtual keypad 113 is long pressed, a guide sound for the corresponding key may be output and the running application may be set to stop. In addition, from the list of various functions displayed on the screen as shown in FIG. 5 , the user can select a desired function by pressing the 'direction key' of the first virtual keypad 113 . At this time, the portable terminal 10 guides the function to be selected by voice whenever the direction key is pressed.

The palpation sheet 30 is preferably attached to only a portion of the touch screen 111 , which is intended to use the remaining part of the touch screen 111 for displaying applications or for direct touch for other purposes. For example, if a circle is drawn on the touch screen 111 after the power is turned on, a function of unlocking the screen may be used.

In the case of the present invention, the UI for the visually impaired may be automatically executed when the lock screen is released without the need to start the application for the visually impaired.

2nd Example

6 is a view showing a palpation sheet 32 according to a second embodiment of the present invention, and FIGS. 7 to 11 are a portable terminal ( 10) is shown.

6, the facilitation sheet 32 according to the second embodiment of the present invention is similar to the facilitation sheet 30 of the first embodiment, but the position, arrangement, and shape of the protrusions are slightly different. These protrusions also have shapes such as ｜, │, ―, ＝, ∧, ∨, ∧, ∨, ∧, ∨, ∧, ∨, ?, ?

The first and second virtual keypads 123 and 125 according to the second embodiment are similar to the virtual keypads 113 and 115 of the first embodiment in terms of function and usage, but the positions, arrangement, and combination of keys are different.

The first and second virtual keypads 123 and 125 shown in FIG. 7 are used as launcher keypads. By using the launcher keypad, the user can directly launch the desired application or application program.

8 shows an example of the first and second virtual keypads 123 and 125 in the Korean mode, and FIG. 9 shows an example of the first and second virtual keypads 123 and 125 in the foreign language mode, and FIG. shows an example of the first and second virtual keypads 123 and 125 in the symbol mode, and FIG. 11 shows an example of the first and second virtual keypads 123 and 125 in the numeric mode.

Like the first embodiment, the key combination of the first virtual keypad 123 is not changed during use, but the key combination of the second virtual keypad 125 can be changed during use. The process and method of changing the key combination of the second virtual keypad 125 are the same as those of the first embodiment.

FIG. 12 shows screens that are executed sequentially when the 'phone' key is selected on the launcher keypad of FIG. 7 . After selecting the 'Phone' key, if the '1' number key of the second virtual keypad 125 in the numeric mode is continuously selected, the phone -> recent calls -> details of recent calls can be checked in sequence.

In the foregoing, the present invention has been shown and described in connection with preferred embodiments for illustrating the principles of the present invention, but the present invention is not limited to the construction and operation as shown and described as such. Rather, it will be apparent to those skilled in the art that many changes and modifications can be made to the present invention without departing from the spirit and scope of the appended claims.

10: portable terminal 30, 32: palpation sheet
110: input unit 111: touch screen
113, 123: first virtual keypad 115, 125: second virtual keypad
120: communication unit 130: output unit
140: memory 170: control unit

Claims (10)

a touch screen;
A first virtual keypad for controlling a screen and an application and a controller for controlling the touch screen to display a second virtual keypad for inputting characters, numbers and symbols on the touch screen to which a palpation sheet is attached,
The control unit controls the touch screen to display the first and second virtual keypads on an area of the touch screen that matches the attached palpation sheet, and based on attribute information of an application running on the touch screen, the second and controlling the touch screen to display one of preset key combinations of a virtual keypad. The method of claim 1,
The control unit is
and determining the positions of the first and second virtual keypads to be displayed on the touch screen according to stored keypad information. The method of claim 1,
The control unit is
and determining a key combination of the first and second virtual key paddles to be displayed on the touch screen according to stored keypad information. The method of claim 1,
The control unit is
requesting the user to touch at least one of the protrusions formed on the palpation sheet by using the guidance voice or the guidance screen,
The portable terminal, characterized in that for determining the attachment position of the palpation sheet and the type of the palpation sheet according to the position of the protrusion touched by the user. 5. The method of claim 4,
The control unit is
The portable terminal, characterized in that the attachment position of the palpation sheet and the type of the palpation sheet are determined by comparing the position of the protrusion touched by the user with pre-stored keypad information. 5. The method of claim 4,
The control unit is
The portable terminal of claim 1, wherein the positions of the first and second virtual keypads to be displayed on the touch screen are determined according to the attachment position of the palpation sheet and the type of the palpation sheet. 5. The method of claim 4,
The control unit is
The portable terminal, characterized in that the size of the palpation sheet and the combination and position of all the protrusions included in the palpation sheet are determined according to the position of the protrusion touched by the user. The method of claim 1,
The control unit is
and when one of the preset key combinations of the second virtual keypad is displayed, the output unit is controlled to output a guide voice explaining the displayed key combination. The method of claim 1,
The control unit is
When the user's drag touch is detected on the second virtual keypad, the portable terminal according to claim 1, wherein a key combination of the second virtual keypad is changed according to the drag touch. The method of claim 1,
The control unit is
and controlling the touch screen to display the application on the remaining area of the touch screen except for the area to which the palpation sheet is attached.

Images