KR20170128196A - Method, Device, and Computer-Readable Medium for Controlling Tactile Interface Device - Google Patents

Abstract

The present invention relates to a method, a device, computer-readable medium for controlling a tactile interface device. More specifically, the present invention relates to a method, a device, and computer-readable medium for controlling a new concept of a tactile interface device, which allows a visually impaired person to more intuitively and efficiently interact with a computer by using a tactile sense in a manner similar to a computer usage method of the public. The method for controlling a tactile interface device comprises: an input conversion step of converting an input from the tactile interface device into an input to an application; and an output conversion step of converting an output from the application into an output of the tactile interface device.

Description

METHOD, APPARATUS AND COMPUTER-READABLE MEDIUM FOR CONTROLLING Tactile Interface Device [0001]

The present invention relates to a method, an apparatus and a computer-readable medium for controlling a tactile interface apparatus, and more particularly, to a method and apparatus for controlling a tactile interface apparatus in which a blind person uses a tactile sense more intuitively and efficiently, To a method, apparatus and computer-readable medium for controlling a tactile interface device of a new concept allowing interaction with a tactile interface device.

In the information society, it is essential to acquire and use information using computers, and this tendency is the same for the general public as well as the visually impaired.

In order to improve the level of informatization of the visually impaired, they should also be able to acquire and use information at a level similar to that of the normal person. This is important because it makes the daily life of the visually impaired more convenient, Do. In addition, it can be said that it is very important because it can ultimately improve the welfare of the visually impaired by providing various education opportunities and expanding opportunities for social advancement and participation.

However, the conventional method of using the present computer mainly uses a method of visually recognizing visual information output through a monitor and inputting information using an input tool such as a keyboard, a mouse, or a touch pad. Therefore, there is a great limitation in visually impaired persons who can not use the time to perceive the output information of the computer at the same level as the general person and to interact such as immediate information input. As a result, the visual impairment significantly degrades the efficiency of computer use, thereby greatly depriving the computer of opportunities for information acquisition and utilization.

Various techniques have been developed to recognize the visual information and to interact with the computer through the use of the hearing, tactile sense, etc., in order to solve the difficulties in using the computer of the visually impaired. As a representative technology, there is a screen reader which assists the visually impaired to use the computer through the hearing. This is a device or software that helps users to use the computer by outputting the contents displayed on the computer screen and the keyboard information inputted by the user.

However, the screen reader has difficulty in finding the screen output information because it searches Graphic User Interface (GUI) elements of the output screen by only one line of linearized information without two-dimensional spatial position information on the output screen. Especially, The more information there is, the greater the difficulty. In addition, since screen readers provide only a simple textual explanatory description for various graphic information such as pictures and diagrams in addition to characters and GUI elements, the visually impaired can understand and interact with graphic information It will have great difficulty.

Another related art is a Braille information terminal which transmits character information through a braille cell by a tactile sense. This is a method that is used as an independent device by providing some functions of a computer useful for a visually impaired person and a method of using as a screen output auxiliary device for outputting text information of a computer screen analyzed by a screen reader in braille. Instead of acting as an interface for efficient interaction with the computer, both of them serve as substitute devices that perform only a few limited functions of the computer as a substitute for the computer, or as output auxiliary devices that output text information in braille do. Particularly, the Braille information terminal specialized in the output of braille has a problem that it can not display the graphic information like the screen reader.

Korean Published Patent No. 10-2012-0063982

It is an object of the present invention to provide a method, an apparatus, and a computer-readable medium for controlling a new concept tactile interface device that enables a visually impaired person to interact with a computer more intuitively and efficiently using a tactile sense in a manner similar to that of a general- Readable medium.

According to an aspect of the present invention, there is provided an electronic device electrically connected to a tactile interface device capable of interacting with a user, the electronic device including: an application executing in the electronic device; And an input / output controller for controlling input / output of the tactile interface device, wherein the tactile interface device can provide a tactile graphic by a tactile sense, and the input / output control portion includes an input conversion portion for converting input from the tactile interface device to input to the application, ; And an output conversion section for converting an output from the application into an output of the tactile interface device.

In the present invention, the input conversion unit may include: a key conversion unit that converts a key input input from the tactile interface device into a form that can be input to the application; And a gesture determination unit for converting the touch input input from the tactile interface device into a form that can be input to the application.

In the present invention, the gesture determination unit may determine whether the touch input corresponds to one of predetermined gestures, and may convert the touch input into a command corresponding to the identified gesture when the gesture is determined.

In the present invention, the input conversion unit may further include a coordinate mapping unit for converting the input coordinates input from the tactile interface device into input coordinates in the application.

In the present invention, the output conversion unit may include a tactile graphics converting unit for converting a focus area including a part of the graphic display output from the application into tactile graphic data, and the tactile graphics provided by the tactile interface device may include Tactile graphic data.

In the present invention, the output conversion unit may further include a TTS conversion unit that converts information output from the application into voice information.

In the present invention, the focus area may be determined by at least one of the previous inputs by the user.

In the present invention, the tactile graphic conversion unit may divide the focus area into a plurality of focus part areas, and convert the display elements of the respective focus part areas into tactile graphic elements according to a predetermined conversion rule, Data can be generated.

In the present invention, the tactile graphics may include a plurality of layers, and one of the plurality of layers may display a tactile icon for executing a command to change a screen of the graphic display output from the application.

In the present invention, the tactile graphics may include a plurality of layers, and the plurality of focus partial areas may correspond to the plurality of layers.

According to an aspect of the present invention, there is provided an electronic device electrically connected to a tactile interface device capable of interacting with a user, the electronic device including: an application executing in the electronic device; And an output transformer for transforming the output from the application into an output of the tactile interface device, wherein the tactile interface device is capable of providing tactile tactile graphics.

In the present invention, the output conversion unit may include a tactile graphics converting unit for converting a focus area including a part of the graphic display output from the application into tactile graphic data, and the tactile graphics provided by the tactile interface device may include Tactile graphic data.

In the present invention, the focus area may be determined by at least one of the previous inputs by the user.

In the present invention, the tactile graphic conversion unit may divide the focus area into a plurality of focus part areas, and convert the display elements of the respective focus part areas into tactile graphic elements according to a predetermined conversion rule, Data can be generated.

In the present invention, the tactile graphics may include a plurality of layers, and one of the plurality of layers may display a tactile icon for executing a command to change a screen of the graphic display output from the application.

In the present invention, the tactile graphics may include a plurality of layers, and the plurality of focus partial areas may correspond to the plurality of layers.

According to an aspect of the present invention, there is provided a method of controlling a tactile interface device electrically connected to an electronic device including an application executed internally and capable of interacting with a user, An input conversion step of converting an input of the application into an input to the application; And an output conversion step of converting an output from the application into an output of the tactile interface device, wherein the tactile interface device is capable of providing a tactile tactile graphics.

In the present invention, the input conversion step may include: an input information receiving step of receiving input information input from the tactile interface device; An input information category discrimination step of discriminating a category of the input information; And an input information converting step of converting the input information into a form that can be input to the application according to the category.

In the present invention, the category may include at least one of a key input and a touch input.

In the present invention, when the category of the input information is a touch input, the input information conversion step determines whether or not the touch input is matched with any one of a plurality of predefined gestures, and when it is matched, , The input information can be converted into a command that can be input to the application corresponding to the matched gesture.

In the present invention, when the category of the input information is a touch input, the input information conversion step determines whether or not the touch input corresponds to the input of the input coordinates, and when the touch input corresponds to the input coordinates, The input coordinates input from the interface device can be converted into input coordinates in the application.

In the present invention, the output conversion step may include a tactile graphic conversion step of converting a focus area including a part of the graphic display output from the application into tactile graphic data.

In the present invention, the haptic graphic conversion step may include: a focus area determination step of determining a focus area including a part of the graphic display output from the application; A focus partial area dividing step of dividing the focus area into a plurality of focus partial areas; A tactile graphic element conversion step of converting a display element of each focus partial area into a tactile graphic element according to a predetermined conversion rule; And a tactile graphic data generation step of generating tactile graphic data including the tactile graphic element.

In the present invention, the tactile graphics may include a plurality of layers, and one of the plurality of layers may display a tactile icon for executing a command to change a screen of the graphic display output from the application.

In the present invention, the tactile graphics may include a plurality of layers, and the plurality of focus partial areas may correspond to the plurality of layers.

In order to solve the above problems, A method for converting a graphical display output by an application of an electronic device into a tactile graphics by a tactile display, the method comprising: converting a focus area including a portion of a graphic display output from the application into tactile graphics data; And provides a method for converting.

In the present invention, the method may further include: a focus area determination step of determining a focus area including a part of the graphic display output from the application; A focus partial area dividing step of dividing the focus area into a plurality of focus partial areas; A tactile graphic element conversion step of converting a display element of each focus partial area into a tactile graphic element according to a predetermined conversion rule; And a tactile graphic data generation step of generating tactile graphic data including the tactile graphic element.

In the present invention, the tactile graphics may include a plurality of layers, and at least one of the plurality of layers may display a tactile icon for executing a command to change a screen of the graphic display output from the application.

In the present invention, at least one of the plurality of layers may display detailed information about tactile graphic elements output from the application in braille form.

In the present invention, the tactile graphics may include a plurality of layers, and the plurality of focus partial areas may correspond to the plurality of layers.

In order to solve the above problems, the present invention provides a computer-

The computer-readable medium stores instructions for causing a computer to perform the steps of: receiving input from a tactile interface device capable of interacting with a user to an application executing on the electronic device; An input conversion step of converting into an input; And an output conversion step of converting an output from the application into an output of the tactile interface device.

In the present invention, the application may be an application capable of performing graphic input, editing, and storage functions.

In order to solve the above problems, 22. A computer-readable medium,

The computer-readable medium stores instructions that cause a computer to perform the steps of: receiving a focus area including a portion of a graphical display output from an application executing within an electronic device, To a computer-readable medium.

The method, apparatus, and computer-readable medium for controlling a tactile interface device in accordance with the present invention can exert an effect of visually impaired persons intuitively recognizing tactile display items in real time.

In addition, the method, apparatus and computer-readable medium for controlling the tactile interface device according to the present invention allow the visually impaired to recognize the information tactually and at the same time instantly input based on the perceived information .

In addition, the method, apparatus and computer-readable medium for controlling the tactile interface device according to the present invention can be applied to a tactile display capable of recognizing tactile and gesture- So that it is possible to promptly and intuitively input information to the user, so that the computer can be effectively used in a manner very similar to that of the general public.

Further, the method, apparatus, and computer-readable medium for controlling the tactile interface device according to the present invention can exhibit the effect of being easily expandable to use various functions of the OS and various application software.

In addition, the method, apparatus and computer-readable medium for controlling the tactile interface device according to the present invention can provide a visually impaired person with a speedy and convenient operation speed that is remarkably improved in comparison with a device such as a screen reader of the related art .

In addition, the method, apparatus, and computer-readable medium for controlling the tactile interface device according to the present invention can exert the effect that the visually impaired persons can conveniently perform graphic related tasks that are difficult to perform with a screen reader.

Further, according to a method, an apparatus and a computer-readable medium for controlling a tactile interface device according to the present invention, WINDOWS ^TM Programs such as 'Paint', which is an auxiliary program, are also available for the visually impaired. The blind can input a picture including the line through the second layer, recognize the picture drawn by himself / herself using the tactile sense in real time, and continue to complete the picture based on the recognized picture information .

Further, the method, apparatus and computer-readable medium for controlling the tactile interface device according to the present invention significantly improve the ability of the visually impaired to utilize the computer, and to expand the market for computer hardware and software for the visually impaired .

1 is a schematic view illustrating an internal structure of a tactile interface device and a user terminal according to an exemplary embodiment of the present invention.
FIG. 2 is a schematic view illustrating an internal structure of a tactile display device according to an exemplary embodiment of the present invention. Referring to FIG.
FIG. 3 is a three-dimensional view illustrating a tactile interface device according to an embodiment of the present invention.
4 is a three-dimensional view illustrating a tactile display unit according to an embodiment of the present invention.
5 is a diagram illustrating examples of a pin driving module according to an embodiment of the present invention.
6 is a cross-sectional view of an example of a pin drive module according to an embodiment of the present invention.
7 is a view schematically showing a plane of an input sensing unit according to an embodiment of the present invention.
8 is a schematic view illustrating a side surface of an input sensing unit according to an embodiment of the present invention.
9 is a diagram schematically illustrating steps of a tactile interface method according to an embodiment of the present invention.
10 is a view schematically showing detailed steps of a tactile display step according to an embodiment of the present invention.
11 is a diagram schematically illustrating detailed steps of an input sensing step according to an embodiment of the present invention.
12 is a diagram showing an example of using a tactile interface device according to an embodiment of the present invention.
13 is a view schematically showing an example of connection between an electronic device for controlling a tactile interface device according to an embodiment of the present invention and a tactile interface device.
14 is a view schematically showing an internal structure of an electronic device for controlling a tactile interface device according to an embodiment of the present invention.
15 is a diagram schematically illustrating the steps of a method for controlling a tactile interface device according to an embodiment of the present invention.
16 is a diagram schematically illustrating the steps of an input conversion step according to an embodiment of the present invention.
17 is a view schematically showing the processes of the output conversion step according to an embodiment of the present invention.
18 is a view schematically showing the steps of the tactile graphic conversion step according to an embodiment of the present invention.
19 is a view schematically showing a tactile graphics according to an embodiment of the present invention.
20 is a table showing a table of tactile icons according to an embodiment of the present invention.
21 is a diagram showing a first example of a display screen of the electronic device.
FIG. 22 is a diagram showing a tactile graphics outputted by converting a first example of the display screen shown in FIG. 21 into a tactile graphics according to an embodiment of the present invention.
23 is a diagram showing a second example of a display screen of the electronic device.
FIG. 24 is a diagram showing a tactile graphics output by converting a second example of the display screen shown in FIG. 23 into a tactile graphics according to an embodiment of the present invention.
FIG. 25 is a view schematically showing a process of drawing a picture by a blind person with a tactile interface device according to an embodiment of the present invention.

Various embodiments and / or aspects are now described with reference to the drawings. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of one or more aspects. However, it will also be appreciated by those of ordinary skill in the art that such aspect (s) may be practiced without these specific details. The following description and the annexed drawings set forth in detail certain illustrative aspects of one or more aspects. It is to be understood, however, that such aspects are illustrative and that some of the various ways of practicing various aspects of the principles of various aspects may be utilized, and that the description set forth is intended to include all such aspects and their equivalents.

In addition, various aspects and features will be presented by a system that may include multiple devices, components and / or modules, and so forth. It should be understood that the various systems may include additional devices, components and / or modules, etc., and / or may not include all of the devices, components, modules, etc. discussed in connection with the drawings Must be understood and understood.

As used herein, the terms "an embodiment," "an embodiment," " an embodiment, "" an embodiment ", etc. are intended to indicate that any aspect or design described is better or worse than other aspects or designs. . As used herein, the terms 'component,' 'module,' 'system,' 'interface,' and the like generally refer to a computer-related entity and include, for example, hardware, It can mean software.

In addition, the term "or" is intended to mean " exclusive or " That is, it is intended to mean one of the natural inclusive substitutions "X uses A or B ", unless otherwise specified or unclear in context. That is, X uses A; X uses B; Or when X uses both A and B, "X uses A or B" can be applied to either of these cases. It should also be understood that the term "and / or" as used herein refers to and includes all possible combinations of one or more of the listed related items.

It is also to be understood that the term " comprises "and / or" comprising " means that the feature and / or component is present, but does not exclude the presence or addition of one or more other features, components and / It should be understood that it does not.

It is also to be understood that the singular forms "a" and "an" above, which do not expressly state otherwise in this specification, include plural representations. Thus, in one example, a " component surface " includes one or more component surfaces.

Also, terms including ordinal numbers such as first, second, etc. may be used to describe various elements, but the elements are not limited to these terms. The terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component. And / or < / RTI > includes any combination of a plurality of related listed items or any of a plurality of related listed items.

Also, the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, the terms "comprises" or "having" and the like refer to the presence of stated features, integers, steps, operations, elements, components, or combinations thereof, But do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, or combinations thereof.

Furthermore, in the embodiments of the present invention, all terms used herein, including technical or scientific terms, unless otherwise defined, are intended to be inclusive in a manner that is generally understood by those of ordinary skill in the art to which this invention belongs. Have the same meaning. Terms such as those defined in commonly used dictionaries are to be interpreted as having a meaning consistent with the contextual meaning of the related art and, unless explicitly defined in the embodiments of the present invention, are intended to mean ideal or overly formal .

Tactile interface device

Hereinafter, an example of a tactile-sensing interface device to be controlled in a method, an apparatus, and a computer-readable medium according to the present invention will be described.

1 is a schematic view illustrating an internal structure of a tactile-sensing device 1000 and a user terminal according to an exemplary embodiment of the present invention.

As shown in FIG. 1, the tactile interface device 1000 is connected to the user terminal A in a pneumatic or wireless manner, and is configured to transmit and receive data. For example, And may receive the graphic signal from the terminal A to provide tactile information. However, unlike FIG. 1, the present invention differs from FIG. 1 in that the tactile interface apparatus 1000 itself operates by a proprietary operating system without an external user terminal, and is operated by a tactile interface apparatus (1000) may be operated. However, also in this case, the tactile-sensing interface apparatus 1000 may be provided with a communication function.

Meanwhile, the user terminal A includes a smart phone, a tablet, a personal computer (PC), a mobile phone, a video phone, an electronic book An e-book reader, a desktop PC, a laptop PC, a netbook PC, a personal digital assistant (PDA) A portable multimedia player (PMP), an MP3 player, a mobile medical device, a camera, a wearable device (for example, a portable multimedia player (PMP) A head-mounted device (HMD), an electronic garment, an electronic bracelet, an electronic necklace, an electronic app apparel, an electronic tattoo, or a smart watch (smart watch) or the like.

1 or 3, the tactile sensing apparatus 1000 according to an exemplary embodiment of the present invention includes an external device connection unit 1100, a tactile display unit 1200, an input sensing unit 1300, a keyboard unit 1400, a control unit 1500, and a speaker unit 1600. The tactile display unit 1200 converts a visual graphic output from a computer to a monitor into a tactile graphic and outputs the visual graphic.

The input sensing unit 1300 may be coupled to the tactile display unit 1200. The input sensing unit 1300 recognizes the tactile information output from the tactile display unit 1200 or the finger touch of the visually impaired person with respect to the tactile graphic and various predetermined gestures and transmits the input signal to the computer. Preferably, the keyboard unit 1400 is preferably a braille keyboard that converts character input signals to a computer by converting braille characters generally used by the visually impaired to general characters.

The braille keyboard transmits the braille to the user terminal connected to the tactile interface apparatus 1000 or the tactile interface apparatus 1000, as in the conventional braille keyboard. Braille keyboard consists of 1 ~ 6 points, backspace, space, and enter button. Braille consists of several dots to form one letter, so the braille keyboard can transmit information of pressed buttons at the same time. The transmitted braille information is transferred to the general character through the software in the tactile interface apparatus 1000 or the software of the user terminal.

Therefore, for the computer input / output function for the visually impaired, the tactile display unit 1200 plays the same role as the monitor of the general computer, and the input sensing unit 1300 functions as the touch screen of the mouse or the tablet PC The keyboard unit 1400, or preferably the braille keyboard serves as a keyboard of a general computer.

FIG. 2 is a schematic view illustrating an internal structure of a tactile display device 1000 according to an embodiment of the present invention.

The tactile interface apparatus 1000 according to an embodiment of the present invention provides tactile information to a user through a plurality of pins based on data received from a connected external device or data generated in the tactile interface apparatus 1000 A tactile display unit 1200; An input sensing unit 1300 for sensing a user input; An external device connection unit 1100 that can be connected to an external device; A keyboard unit 1400 for receiving information from a user in a key input form; A speaker unit 1600 capable of outputting sounds to the outside; And a control unit 1500 for controlling operations of the tactile display unit 1200, the input sensing unit 1300, the external device connection unit 1100, the keyboard unit 1400, and the speaker unit 1600 .

The tactile display unit 1200 provides tactile information with at least one dimensional tactile pixel, and the tactile pixel is constituted by a plurality of pins moving up and down by applying power to a transducer including a piezoelectric ceramics and an elastic body . Preferably, the tactile pixel provides tactile information in two dimensions.

The tactile display unit 1200 includes a display data receiving unit 1210 that receives data received from an external user terminal or receives data generated in the tactile interface apparatus 1000; A tactile data converter 1220 for converting the data into tactile display data; And a plurality of pin driving modules (1230) driven by the tactile display data; And a driving power supply unit 1240 which is supplied with power for driving the tactile display unit 1200, and provides tactile information or tactile graphics based on the input data.

Meanwhile, the input sensing unit 1300 emits a plurality of lights, senses a plurality of emitted lights, and when the sensing value of the light emitted by the physical interception changes, the changed sensing value From the position information of the light having the light source.

Specifically, the input sensing unit 1300 includes a position sensing module 1310; And a positioning unit 1320. The position sensing module 1310 includes a plurality of light emitters and a plurality of light receiving sensors, and the positioning unit 1320 receives the sensing data from the plurality of light receiving sensors Sensing user input.

Preferably, the light used in the input sensing unit 1300 is infrared rays, and the light is emitted in a form of irradiating a certain cross section.

The input sensing unit 1300 may be disposed adjacent to, or more preferably, coupled to the tactile display unit 1200 or the pin driving module 1230 of the tactile display unit 1200. Accordingly, the user can recognize the tactile information or the tactile graphic provided from the tactile display unit 1200 as a tactile sense, and can perform an input, a gesture or the like directly on the tactile display.

However, when the user performs input to the input sensing unit 1300, the finger of the user is strongly contacted to the pin driving module 1230 of the tactile display unit 1200 to damage the pin driving module 1230 The position sensing module 1310 of the input sensing unit 1300 may be spaced apart from the pin driving module 1230. [ Alternatively, the area that the user can input to the input sensing unit 1300 is preferably spaced apart from the operating area of the pin driving module 1230. Alternatively, the area that the user can input to the input sensing unit 1300 is preferably arranged at a height higher than the height when the pin driving module 1230 is lifted.

Preferably, the plurality of light emitters are arranged along a first axis and a second axis, and the plurality of light receiving sensors are arranged along a first axis and a second axis, and the plurality of light receiving sensors include the plurality Is arranged so as to correspond to the light emitter of the light source.

The external device connection unit 1100 includes at least one of a communication module unit 1110 capable of performing wireless communication and a wired connection unit 1120 capable of wiredly connecting with the external device. The communication module 1110 includes a Bluetooth communication module, a Zigbee communication module, an infrared communication module, a Bluetooth low energy (BLE) communication module, an audio communication module, an LTE (long term evolution) A communication module, an IrDA-based infrared communication module, a wireless LAN (WLAN), a WiBro module, and a wireless USB (Wireless USB) module. The wired connection unit 1120 may include a connection module using a universal serial bus (USB) interface, and may be a wired connection module capable of transmitting and receiving data.

Meanwhile, the speaker unit 1600 may include a function of reading the tactile display information, character information, or information output in response to the information input by the input sensing unit 1300 by voice. Can be performed.

The controller 1500 controls the overall operation of the tactile interface device, the external device connection unit 1100; The keyboard unit 1400; Or controls the operation of the speaker unit 1600. For example, when data is received from an external user terminal, the data is transmitted to the display data receiving unit 1210 of the tactile display unit 1200, and the operation of the tactile display unit 1200 is commanded. Alternatively, when a user's input is input from the input sensing unit 1300, the control unit 1500 processes the input or transmits the input signal to the user terminal A connected through the external device connection unit 1100.

If the tactile display device is not connected to an external user terminal but operates with its own OS and applications, it can operate independently including a CPU and a memory device of a level that can perform its own operation. Even if it is operable independently, the tactile display device may perform communication with the outside.

FIG. 3 is a three-dimensional view of a tactile display device 1000 according to an embodiment of the present invention.

3, the tactile display apparatus 1000 according to the present invention includes components of the tactile display unit 1200 exposed to the outside, in particular, a plurality of pin drive modules 1230, Lt; / RTI > With the tactile display unit 1200, the user can tactilely recognize the plurality of pin drive modules 1230 of the tactile display unit 1200.

Meanwhile, the user can input a command or the like through touch input through the input sensing unit 1300 while recognizing the tactile display tactilely. Herein, the 'touch input' is not interpreted in a narrow sense and includes both an input for input while slightly touching the uppermost pin of the pin driving module of the tactile display unit and an input for not touching the uppermost pin of the pin driving module do.

The user's finger is strongly contacted to the pin drive module 1230 of the tactile display unit 1200 and the pin drive module 1230 is damaged The input sensing unit 1300 according to the present invention is different from the capacitive sensing method in which a change in electric capacity of the tactile pin surface used in a general touch display is detected, A non-contact recognition method based on optical sensing, more preferably, infrared sensing is introduced so that operation, input, gesture, etc. can be recognized regardless of a touch with respect to the touch. That is, preferably, the input sensing unit is performed in a non-contact manner.

In this specification, non-contact sensing or non-contact sensing refers to a sensing method in which a sensing element or sensor can recognize a physical operation or a physical form even if the sensing element does not physically contact the sensing element or sensor, As an example of this, an embodiment of the present invention uses an infrared sensing based recognition method.

Also, as shown in FIG. 3, the position sensing module 1310 of the input sensing unit 1300 may be spaced apart from the pin driving module 1230. Alternatively, the area that the user can input to the input sensing unit 1300 is preferably spaced apart from the operating area of the pin driving module 1230. Alternatively, the area that the user can input to the input sensing unit 1300 is preferably arranged at a height higher than the height when the pin driving module 1230 is lifted.

Meanwhile, the user can input information to the tactile interface apparatus 1000 through the keyboard unit 1400, for example, to input text information in order to input specific information.

In addition, in order to output voice or sound information of text to the user, the tactile sensing interface apparatus 1000 may output voice information to the user through the speaker unit 1600. [

3, the tactile sensing apparatus 1000 may be connected to an external user terminal through a wired connection unit 1120 or may be connected to an external user terminal by wireless communication through a communication module unit 1110. [ In this case, the GUI elements in the user terminal are tactually displayed in the tactile interface apparatus 1000, and the user can input a response or input to the GUI elements through the input sensing unit 1300 or the keyboard. Since the tactile display device 1000 can output according to the tactile display unit 1200 or the speaker unit 1600 according to a command input by the user through the input sensing unit 1300, In the case of the disabled, a user terminal such as a PC based on a graphic screen can be used more intuitively.

In addition, the tactile interface apparatus 1000 is not limited to simply recognizing the tactile information, but can recognize the tactile information and intuitively input commands or the like to the tactile interface apparatus 1000 As a result, it is possible to use a user terminal such as a PC based on a graphic screen as a convenience to the general public.

4 is a three-dimensional view illustrating a tactile display unit 1200 according to an embodiment of the present invention.

The tactile display unit 1200 may display or provide tactile pixels of one or more dimensions. FIG. 4 shows an example of a tactile display unit 1200 capable of providing two-dimensional tactile pixels. The tactile pixel is constituted by a plurality of pins moving up and down by applying power to a transducer including a piezoelectric ceramic and an elastic body.

Specifically, the tactile pixel is represented by a plurality of pin driving module groups, and each of the pin driving module groups is made up of a plurality of pin driving modules 1230. Alternatively, the entire pin drive module may constitute one pin drive module group.

Specifically, the tactile display unit 1200 includes a master board 1250; A display data receiving unit (1210) electrically connected to the master board (1250) and receiving primary data for tactile display; A tactile data converter 1220 for converting the primary data received from the display data receiver 1210 into data for tactile display; A plurality of slave boards 1260 electrically connected to the master board 1250; And a plurality of pin driving modules (1230) disposed on an outer surface of each of the plurality of slave boards (1260); And a driving power supply unit 1240 for supplying driving power to the above components through the master board 1250.

The driving power supply unit 1240 preferably receives power supplied from the outside to the master board 1250.

The tactile data converting unit 1220 includes a first tactile data converting unit 1221 arranged on the master board 1250, And a second tactile data converter 1222 disposed in the slave board 1260. The first tactile data converter 1221 converts the primary data into tactile data for each slave board 1260, And the second tactile data converting unit 1222 converts the secondary data into third data for driving a group of the plurality of pin driving modules 1230 disposed in each slave board 1260, Data.

The tactile display unit 1200 converts a visual graphic into a tactile graphic form by vertically moving a pin of the pin driving module 1230 having a plurality of arrangements. Since the tactile display unit 1200 according to the present invention does not require the configuration of an additional module as compared with the relay mechanism based on the piezoelectric bimorph in which the pin rises when the voltage is applied and the pin is lowered when the voltage is not applied , It is possible to miniaturize and has a structure in which multiple arrangements are easy.

Therefore, the tactile display unit 1200 as in the present invention can exhibit the effect of providing a high-resolution tactile display at the same time while reducing the size of the apparatus.

The master board 1250 receives the tactile graphic data input through the display data receiving unit 1210 and transmits the tactile graphic data to the first tactile data converting unit 1221. The display data receiving unit 1210 may be a universal asynchronous receiver / transmitter (UART), and the first tactile data converting unit 1221 may be an MCU (Micro Controller Unit).

The first tactile data converting unit 1221 analyzes the tactile graphic data input through the display data receiving unit 1210 and transmits the tactile graphic data to a plurality of slave boards 1260. In such a transmission, the first tactile data converter 1221 may convert data based on the information of the plurality of slave boards 1260, and then transmit the converted data to the slave board 1260.

Alternatively, the first tactile data converter 1221 may convert the tactile graphic data into data corresponding to the respective slave boards 1260, and may transmit the converted data to the respective slave boards 1260 It is possible.

Alternatively, the tactile graphic data transmitted through the display data receiving unit 1210 may be directly transmitted to the slave board 1260 without the first tactile data converting unit 1221.

Then, the second tactile data converter 1222 incorporated in the slave board 1260 converts the data received from the first tactile data converter 1221 or the data received from the first tactile data converter 1221 using a previously programmed CPLD (Complex Programmable Logic Device) The data received through the display data receiving unit 1210 is divided according to each pin driving module array composed of the pin driving module 1230 or the plurality of pin driving modules 1230, Or to a pin drive module array composed of a plurality of pin drive modules 1230.

According to the operation signal, the pin of the pin driving module 1230 can move up and down. Preferably, the pin interval and the rising height are in the range of 1 mm to 10 mm.

5 is a diagram illustrating examples of the pin driving module 1230 according to an embodiment of the present invention.

Hereinafter, an embodiment of the pin drive module 1230 shown in Figs. 5A and 5B will be described.

As shown in Fig. 5 (A), the pin drive module 1230 includes a transducer 1231 that converts a vibration motion to a linear motion; A shaft 1232 connected to the transducer; A moving body 1233 which can move up and down in the axial direction of the shaft 1232 on the shaft 1232; And a pin 1234 that can move according to the movement of the moving body 1233. [

Preferably, the transducer 1231 includes a plurality of piezoelectric ceramics and elastic bodies. When the power is applied, the transducer 1231 linearly converts vibrations due to expansion / contraction of the piezoelectric ceramics to move the moving body 1233 up and down , And the movement of the moving body 1233 can be controlled by controlling the transducer.

Hereinafter, an embodiment of the pin drive module 1230 shown in Figs. 5 (C) and 5 (D) will be described.

As shown in FIG. 5C, the pin drive module 1230 includes a transducer 1231, which converts the oscillating motion into a linear motion; A shaft 1232 connected to the transducer; A moving body 1233 which can move up and down in the axial direction of the shaft 1232 on the shaft 1232; And a pin 1234 movable according to the movement of the moving body 1233; And a casing 1235 surrounding the shaft 1232 and having one end of the pin 1234 inserted therein.

Preferably, the transducer 1231 includes a plurality of piezoelectric ceramics and elastic bodies. When the power is applied, the transducer 1231 linearly converts vibrations due to expansion / contraction of the piezoelectric ceramics to move the moving body 1233 up and down , And the movement of the moving body 1233 can be controlled by controlling the transducer.

6 is a cross-sectional view of an example of a pin drive module 1230 according to an embodiment of the present invention.

6, the transducer 1231 of the pin drive module 1230 includes a housing 1231.1, a first piezoelectric ceramic 1231.2, a second piezoelectric ceramic 1231.4 disposed inside the housing 1231.1; And an elastic member 1231.3 disposed between the first piezoelectric ceramic and the second piezoelectric ceramic.

On the other hand, the moving body 1233 includes a moving body 1233.1 and a rubber ring 1233.2 disposed inside the moving body 1233.1.

By controlling the voltages applied to the first piezoelectric ceramic 1231.2 and the second piezoelectric ceramic 1231.4 of the transducer 1231 in this structure, the vertical movement of the moving body 1233 can be efficiently controlled.

7 is a diagram schematically showing a plane of the input sensing unit 1300 according to an embodiment of the present invention.

The input sensing unit 1300 according to an exemplary embodiment of the present invention can perform intuitive interaction with the visually impaired persons and input various information about the tactile graphics output by the tactile display unit 1200. [ In addition, the input sensing unit 1300 may combine the finger touch recognition device with the tactile display unit 1200 to add a finger touch and a finger gesture recognition function.

The tactile display device 1000 according to the present invention displays a tactile graphic by physically raising the tactile pins, unlike a general touch display such as a tablet PC or the like, and displays an excessive pressure The touch tactile graphic or tactile information output may be damaged and confuse the information.

Accordingly, preferably, the input sensing unit 1300 of the present invention is different from the capacitance type sensing a change in electric capacity of a surface of a tactile pin used in a general touch display, A non-contact recognition method based on optical sensing, more preferably, infrared sensing is introduced so that operation, input, gesture, That is, preferably, the input sensing unit is performed in a non-contact manner.

In this specification, non-contact sensing or non-contact sensing refers to a sensing method in which a sensing element or sensor can recognize a physical operation or a physical form even if the sensing element does not physically contact the sensing element or sensor, As an example of this, an embodiment of the present invention uses an infrared sensing based recognition method.

According to the input sensing unit 1300 of the present invention, the infrared beams are designed to be projected slightly higher than the maximum height of the raised tactile cell pins, so that a light touch The touch operation can be recognized at a high speed without damaging the rising state of the pins.

In addition, various touch-based gestures such as swipe and rotation can be recognized without physical hitching of the tactile pins, thereby further diversifying the interaction between the visually impaired and the computer. Another advantage of the input sensing unit 1300 is that it is designed in a detachable manner so that it can be easily combined with other tactile displays.

The input sensing unit 1300 emits a plurality of lights, senses a plurality of emit light, and when the sensed value of the light emitted by the physical interception changes, And senses the user input from the position information of the light.

Here, when the sensed value of the emit light changes, both the case where the emitting light is intercepted and the light is not sensed, as well as the case where the intensity of the emitting light is weakened are all included.

Preferably, the light used in the input sensing unit 1300 is infrared rays, and emits light in the form of irradiating light on a certain cross section.

In the input sensing unit 1300, the input sensing unit 1300 includes a position sensing module 1310; And a positioning unit 1320. The position sensing module 1310 includes a plurality of light emitters and a plurality of light receiving sensors, and the positioning unit 1320 receives the sensing data from the plurality of light receiving sensors Sensing user input.

Specifically, the position sensing module 1310 includes a first axial light emitter 1311, a second axial light emitter 1312, a first axis light receiving sensor 1313, and a second axial light receiving sensor 1314 . A two-dimensional tactile display or a tactile pixel can be realized by the configuration of the light emitter and the light receiving sensor.

The first axis light receiving sensor 1313 is disposed at a position spaced apart from the position of the first axial light emitter 1311 by a predetermined distance in a direction in which the second axial light receiving sensor 1313 emits from the position of the first axial light emitter 1311, Are arranged at a position spaced apart from the position of the biaxial light emitter 1312 by a predetermined distance in a direction to emit.

Preferably, the plurality of first axis light receiving sensors 1313 are disposed so as to correspond to a plurality of first axial light emitters 1311, respectively. For example, when the first, second, third, and fourth light emitters are provided on the first axis, the first light receiving sensor corresponding to the first light emitter, the second light emitter corresponding to the second light emitter, It is preferable that a light receiving sensor 3, a light receiving sensor 3 corresponding to the light emitter 3 and a light receiving sensor 4 corresponding to the light emitter 4 are provided, and one light receiving sensor is preferably provided with a corresponding light emitter It is preferable to be able to sense light. In this arrangement, the number of the first axial light emitters 1311 x the number of the second axial light emitters 1312 becomes the resolution of the tactile pixel capable of sensing.

8 is a view schematically showing a side surface of the input sensing unit 1300 according to an embodiment of the present invention.

As shown in FIG. 8, when the light emit by the light emitter is blocked by the user's finger, the sensed value of the light received is changed in the corresponding light receiving sensor, .

The light emitter and the light receiving sensor of the input sensing unit 1300 may be disposed adjacent to or more preferably in the tactile display unit 1200 or the pin driving module 1230 of the tactile display unit 1200 do. Accordingly, the user can recognize the tactile information or the tactile graphic provided from the tactile display unit 1200 as a tactile sense, and can perform an input, a gesture or the like directly on the tactile display.

However, when the user performs input to the input sensing unit 1300, the finger of the user is strongly contacted to the pin driving module 1230 of the tactile display unit 1200 to damage the pin driving module 1230 The position sensing module 1310 of the input sensing unit 1300 may be spaced apart from the pin driving module 1230. [ Alternatively, the area that the user can input to the input sensing unit 1300 is preferably spaced apart from the operating area of the pin driving module 1230. Alternatively, the area that the user can input to the input sensing unit 1300 is preferably arranged at a height higher than the height when the pin driving module 1230 is lifted.

The maximum height H1 of the pins from the cap layer 1201 having a plurality of holes in which the pins of the pin driving module 1230 can move up and down can be obtained from the cap layer 1201 And more preferably lower than the height H2 from the cap layer to the lowermost region of the light emitted from the light emitter.

9 is a diagram schematically illustrating steps of a tactile interface method according to an embodiment of the present invention.

The tactile interface method capable of interacting with a user according to an embodiment of the present invention is a method of interacting with a user by using first data received from a connected external device or first data generated in the tactile interface device 1000, A tactile display step (SlO) for providing information to a display providing tactile information to a user by means of a pin; And an input sensing step (S20) of sensing a user input in a non-contact manner.

Preferably, the tactile display step and the input sensing step are performed in real time, so that they are not limited to simply recognizing the tactile information by the visually impaired person, but can recognize the tactile information and respond intuitively to the tactile interface device 1000). As a result, it is possible to use a user terminal such as a PC based on a graphic screen as a convenience to the general public.

10 is a view schematically showing detailed steps of a tactile display step according to an embodiment of the present invention.

The tactile display step S10 provides tactile information by tactile pixels represented by a plurality of pins moving up and down by applying power to a transducer including a piezoelectric ceramic and an elastic body.

Specifically, the tactile display step includes a primary data receiving step (S11) of receiving primary data transmitted from the connected external device or primary data generated in the tactile interface device (1000);

A slave board information loading step S12 for loading slave board 1260 information such as the number of slave boards 1260 and the number of tactile pixels that can be expressed by each slave board 1260;

A secondary data conversion step (S13) of converting the primary data into secondary data according to the loaded slave board (1260) information;

A slave board transmitting step (S14) of transmitting the converted secondary data to each slave board (1260);

Each of the slave boards 1260 converts the secondary data into tertiary data as a driving signal of a pin driving module array composed of each pin driving module 1230 or a plurality of pin driving modules 1230, Step S15;

And a pin driving step S16 for driving each pin driving module 1230 with the generated driving signal.

The tactile display step converts the visual graphic into a tactile graphic form through the up and down movement of pins of the pin driving module 1230 composed of multiple arrangements and transmits the tactile graphic. The tactile display step as in the present invention requires no additional module configuration as compared with the relay mechanism using the piezoelectric bimorph structure in which the pin rises when a voltage is applied and the pin is lowered when a voltage is not applied, And can be implemented in a structure in which multiple arrangements are easy.

Therefore, the tactile display step as in the present invention can exhibit the effect of providing a high-resolution tactile display at the same time while reducing the size of the apparatus.

Preferably, the primary data receiving step S11 is performed in the master board 1250 and is performed through a universal asynchronous receiver / transmitter (UART) module, which receives external tactile graphic data.

The slave board information loading step S12 and the secondary data converting step S13 may be performed by an MCU (Micro Controller Unit) electrically connected to the master board 1250. [

Preferably, the secondary data conversion step S13 analyzes tactile graphic data input through the primary data reception step S11. Thereafter, the slave board transmission step S14 transmits the analyzed tactile graphic data to the slave board 1260. [

The secondary data conversion step S13 and the slave board transmission step S14 may be performed by converting the data based on the information of the plurality of slave boards 1260 and transmitting the data to the slave board 1260 Do.

Alternatively, the secondary data conversion step S13 may convert the tactile graphic data into data corresponding to the respective slave boards 1260, and may transmit the converted data to the respective slave boards 1260 have.

Alternatively, the tactile graphic data transmitted through the primary data receiving step S11 may be directly transmitted to the slave board 1260 without the secondary data converting step S13.

Thereafter, the tertiary data conversion step S15 is performed by the processing unit built in the slave board 1260. [ In the tertiary data conversion step, the data received through the received data using a pre-programmed CPLD (Complex Programmable Logic Device) is transmitted to each pin drive module 1230 or each of the plurality of pin drive modules 1230 According to the pin drive module array, and transmits an operation signal to the pin drive module array constituted by the respective pin drive module 1230 or the plurality of pin drive modules 1230. [

The plurality of slave boards 1260 are disposed on a single master board 1250 and are preferably electrically connected to the master board 1250.

11 is a diagram schematically illustrating detailed steps of an input sensing step according to an embodiment of the present invention.

Wherein the input sensing step comprises the steps of emitting a plurality of lights, sensing a plurality of emit light, sensing a user's input in an interfacing device for providing tactile information, When the sensing value of the light changes, the user input is sensed from the position information of the light having the changed sensing value. Such an input sensing method can perform intuitive interaction of the visually impaired persons with respect to the tactile graphic output by the tactile display unit 1200 and various information input. In addition, the input sensing unit 1300 may combine the finger touch recognition device with the tactile display unit 1200 to add a finger touch and a finger gesture recognition function.

More specifically, the input sensing step may include:

A step S21 of emitting light to emit a plurality of lights;

A light sensing step (S22) for sensing light at a position spaced a predetermined distance in a direction from the position where the light is emitted to the direction in which the light is emitted, for each of the plurality of lights;

An optical sensor information loading step (S23) of loading information of the light receiving sensor having a change in the sensing value; And

And a user input calculation step (S24) of calculating a user's input based on the sensed result in the light sensing step.

Meanwhile, the step of meeting light includes a plurality of first axial light emitters 1311 disposed along the first axis and a plurality of second axial light emitters 1312 disposed along the second axis, The light sensing step includes a first axis receiving sensor 1313 corresponding to the plurality of first axial light emitters 1311 and a second axis receiving sensor 1313 corresponding to the plurality of second axial light emitters 1312. [ The sensor 1314 senses a plurality of lights.

The user input calculation step may further include calculating a user input based on a change value of the amount of received light of the plurality of first axis light receiving sensors 1313 and the plurality of second axis light receiving sensors 1314, Lt; / RTI >

The tactile display device 1000 according to the present invention displays a tactile graphic by physically raising the tactile pins, unlike a general touch display such as a tablet PC or the like, and displays an excessive pressure The touch tactile graphic or tactile information output may be damaged and confuse the information.

Therefore, the input sensing step of the present invention is different from the capacitive sensing method of sensing the electric capacity change of the surface of the tactile pin used in a general touch display, so that the touch sensing operation can be recognized on the tactile pins, More preferably, the non-contact recognition method based on infrared sensing is introduced.

According to the input sensing step of the present invention, the infrared beams can be projected to be slightly higher than the maximum height of the raised tactile cell pins. Thus, regardless of the vertical movement of the tactile pins, The touch operation can be recognized at a high speed without damaging the rising state.

In addition, various touch-based gestures such as swipe and rotation can be recognized without physical hitching of the tactile pins, thereby further diversifying the interaction between the visually impaired and the computer.

12 is a diagram showing an example of use of the tactile display device 1000 according to an embodiment of the present invention.

As shown in FIG. 12, according to an embodiment of the present invention, a pin can be raised in real time according to a finger movement, and a free drawing function can be performed freely to draw a graphic, and tactile display or tactile information The user can easily draw a graphic while recognizing the previously drawn graphic.

Accordingly, the tactile display device according to an embodiment of the present invention is a general tactile display device in which a general person of normal vision recognizes visual information through a monitor and inputs information through a mouse (or a touch screen of a tablet PC) In a similar way, the blind can use the tactile sense to interact with the computer more easily and intuitively.

The above-described tactile interface apparatus is described as a tactile interface apparatus that can be used as one example in a method, apparatus, and computer-readable medium for controlling a tactile interface apparatus described later. The description of the method, apparatus, and computer-readable medium is not limited to the above description, but should be interpreted in the best mode according to its own invention.

Method, apparatus and computer-readable medium for controlling a tactile interface apparatus

Hereinafter, a method, an apparatus, and a computer-readable medium for controlling the tactile interface device according to the present invention will be described.

13 is a view schematically showing an example of connection between an electronic device B for controlling a tactile interface device according to an embodiment of the present invention and a tactile interface device 1000. In Fig.

Although the tactile interface device 1000 connected to the electronic device B of the present invention is shown as having the same components as the tactile interface device described above in Fig. 13, this is merely an example, and the present invention is not limited thereto . The tactile interface device connected to the electronic device B controlling the tactile interface device of the present invention described below includes any interface device capable of providing tactile graphics, a tactile interface device capable of interacting with the user, And an electronic device B capable of outputting a graphic.

As shown in Fig. 13, the electronic device B is connected to a tactile interface device, and the connection is a connection including both a wired connection and a wireless connection.

The electronic device B includes the aforementioned user terminal A shown in Fig. Accordingly, the electronic device B may be a smart phone, a tablet, a personal computer (PC), a mobile phone, a videophone, an electronic book An e-book reader, a desktop PC, a laptop PC, a netbook PC, a personal digital assistant (PDA) A portable multimedia player (PMP), an MP3 player, a mobile medical device, a camera, a wearable device (for example, a portable multimedia player (PMP) A head-mounted device (HMD), an electronic garment, an electronic bracelet, an electronic necklace, an electronic app apparel, an electronic tattoo, or a smart watch (smart watch) or the like, and preferably A PC flow. In the electronic device (B) comprises as at least one processor and a memory, and selectively fixed to the display device such as a monitor, or may be provided with its own display module.

Alternatively, the electronic device B may be in combination with the tactile interface device. In this case, the tactile interface device and the electronic device B controlling the tactile interface device may be recognized as a single device. In the case of such a combined device, the electronic device B and the tactile interface device may be in the form of sharing a memory with the processor.

The electronic device B according to the present invention includes an application 3000 executed in the electronic device B and an input / output control unit 2000 for controlling input / output between the tactile interface device and the application 3000.

The application 3000 may include an application (e.g., a searcher in WINDOWS ^TM ) for performing a function or the like fundamentally provided in the OS, an application for performing a function supplementarily provided in the OS (for example, WINDOWS ^TM (Such as MS WORD ^TM ) installed separately from the OS and performing its own functions, applications that perform their own functions that run on the Web, applications on smartphones, It is a term that includes applications that can be used. That is, the application 3000 collectively refers to programs having executable instructions.

A general user can execute the application 3000 on the electronic device B and interact with the application 3000 on a mouse, a keyboard, a monitor, or a touchpad. However, This interaction is made mainly by the graphic information which is difficult to recognize.

Therefore, the inventor of the present invention derives an invention for controlling and controlling the interface (3000) of the application (3000) to be used by the visually impaired person or the input / output of the application (3000) Respectively. In addition, the control method of the tactile interface device of the present invention is not limited to the input / output system using tactile information in other devices such as vehicles and flight devices, and the input / output system . ≪ / RTI >

According to the present invention, the input / output controller 2000 converts the input / output of the application 3000 into a form that can be used by the visually impaired person, implements the input / output in the tactile interface device, (3000) can be more intuitively and conveniently used. Particularly, since the tactile interface device 1000 is a device that transmits information to a user using tactile information and receives information or commands from a user through tactile input or touch input, it is inconvenient for a visually impaired person So that the application 3000 can be used.

14 is a view schematically showing an internal structure of an electronic device B for controlling a tactile interface device according to an embodiment of the present invention. As shown in FIG. 14, the input / output control unit 2000 includes an input conversion unit 2100 that converts an input from the tactile interface device to an input to the application 3000; And an output conversion unit 2200 for converting an output from the application 3000 into an output of the tactile interface device.

Preferably, in the apparatus for controlling a tactile interface apparatus according to an embodiment of the present invention, in addition to the operation of displaying output information from the application 3000 in a tactile display form, ), And input the converted data into the application 3000. [0253] FIG.

That is, according to the present invention, the user does not receive information manually but inputs information intuitively based on the recognized information, and the input / output control unit 2000 can input the input information into the application 3000 So that the user can use the application 3000 at a level similar to that of a general user. The input / output control unit 2000 can use the application 3000 that draws a picture by tactually inputting a blind person as shown in FIG.

The input conversion unit 2100 includes a key conversion unit 2110 for converting a key input input from the tactile interface device into a form that can be input to the application 3000; A gesture discrimination unit 2120 for converting a touch input input from the tactile interface device into a form that can be input to the application 3000; And a coordinate mapping unit 2130 for transforming the input coordinates input by the touch input or the like into the input coordinates in the application 3000 in the tactile interface device.

Preferably, the input converted by the key conversion unit 2110 received from the tactile interface unit is a braille keyboard for converting a braille character generally used by the visually impaired to a normal character and delivering a character input signal to the computer Lt; / RTI >

The Braille keyboard transmits the Braille to the user terminal connected to the tactile interface device or the tactile interface device as well as the conventional braille keyboard. Braille keyboard consists of 1 ~ 6 points, backspace, space, and enter button. Braille consists of several dots to form one letter, so the braille keyboard can transmit information of pressed buttons at the same time.

In this case, the key conversion unit 2110 translates the braille keyboard input into a general character.

Meanwhile, the gesture determination unit 2120 determines whether the touch input corresponds to one of predetermined gestures, and converts the touch input into a command corresponding to the identified gesture when the gesture is determined.

Here, the touch input refers to an input form including all of the input by the non-contact type optical-based sensing shown in FIGS. 7 and 8, the input by the touch recognition by the capacitance change generally used in the tablet PC, do. In other words, the touch input includes not only a key input type but also an input type in which a user directly inputs the touch input.

Here, an example of the preset gesture is shown in Table 1 below.

[Table 1]

Gesture type Application input

Single tap Output the type and text of the GUI element at the touch location in braille and voice

Double tap Functions like double click of mouse

Long tap after tap Functions like right click of mouse

Panning after tap Scroll up / down / left / right

Drawing circle after tap Magnification (clockwise) / reduction (counterclockwise)

The interaction between the tactile interface device and the electronic device is primarily performed through the gesture recognition function of the touch input. For the blind and intuitional computer operation, various touch operations as shown in Table 1 are defined, It is preferable to transfer it as an input signal to the electronic device or the application 3000 of the electronic device.

For example, a single tap gesture outputs the type of GUI (Graphic User Interface) element and corresponding text to the tactile icon at the touched position in braille and voice. For example, a blind person recognizes a tactile icon recognized by a tactile graphics through a swipe operation, and performs a single tap gesture on the tactile icon to move the focus to the corresponding GUI element. In addition, the tab gesture may output the detailed information of the GUI element to be selected, for example, text information in braille to one of the layers displaying tactile graphics of the tactile interface device, and also output it as TTS-based voice.

Long tap, long tap after tap, panning after tap, and drawing circle after tap corresponding to the remaining gestures in Table 1 are performed once, followed by tap, long tap, panning, and drawing the circle gestures . This is so defined that the finger swipe motion for recognizing the tactile graphics of a visually impaired person and the defined gesture are confused and not recognized. The double tap gesture has the same function as the double left click of the mouse that executes the default action of the GUI element by touching twice in the same position quickly. Long tap after tap The gesture is a tap that is touched at the same position and touched continuously for a certain amount of time, and has the same function as a right click of the mouse. Panning after tap The gesture is an operation to perform a panning operation by touching again at the same position after performing a tap once, and has a function of scrolling up / down / left / right according to the panning direction. Drawing circle after tap A gesture is a motion that draws a circle by quickly touching again at the same position after performing a tap once. It has a screen enlargement function in the clockwise direction and a screen reduction function in the counterclockwise direction.

The coordinate mapping unit 2130 is used to convert coordinates or input coordinates in the tactile interface device into coordinates in the application 3000.

As will be described later, when the electronic device is a PC, tactile icons are implemented in the tactile graphics outputted by reconfiguring the GUI elements of the monitor display screen of the PC. Since the tactile icon is not matched with the monitor display screen in a spatial 1: 1 manner, when the input coordinates at which the touch input occurs in the tactile interface device are used as they are, It can not be discriminated by the application 3000.

Preferably, the coordinate mapping function of the coordinate mapping unit 2130 is a function for mapping a tactile icon object currently output as a tactile graph to a GUI element displayed on a monitor display screen when driving an actual application 3000, The table is constructed and the recognized touch coordinates are mapped.

The tactile graphics displayed on the tactile interface device and the display screen when the application 3000 is generally executed may differ in resolution, configuration, and position. In this case, 3000, the input coordinates in the tactile interface must be converted or mapped to the input coordinates in the application 3000, which is performed by the coordinate mapping unit 2130. With the coordinate mapping unit 2130, the user can use the function of the application 3000 at a general user level without reducing or simplifying the function of the application 3000.

For example, in order to execute a command 'A' in the application 3000 (for example, when the user opens the My Documents folder on the Windows desktop), the tactile interface device When performing Double Tab (see Table 1 above), the coordinate mapping unit 2130 maps the coordinates of the point in the tactile interface device to the coordinates of the icon 'A' of the connected application 3000, The 'A' command in the application 3000 can be executed.

As described above, the input conversion unit 2100 includes a touch gesture recognition function for recognizing a gesture with respect to a touch input of the tactile interface device, a coordinate mapping function for indicating a GUI element corresponding to the touch coordinates input from the touch recognition device, It is possible to perform a reverse grayscale function for converting the braille information input from the keyboard into general characters.

The output conversion unit 2200 includes a tactile graphic conversion unit 2210 for converting a focus area including a part of the graphic display output from the application 3000 into tactile graphic data, The tactile graphics provided by the tactile graphics data may be implemented from the tactile graphics data.

In the present specification, the "graphic display" refers to a concept including the visual screen that the application outputs to a general visual display device such as a monitor, a visual element included in the visual screen, or data for implementing them. The output conversion unit 2200 further includes a TTS conversion unit 2220 that converts information output from the application 3000 into voice information. Preferably, the TTS converting unit 2220 performs conversion of the detailed information of the textual form of the tactile element referred to by the input from the tactile interface apparatus into speech.

On the other hand, when the actual application 3000 is driven, when all the GUI elements output to the monitor display screen are converted into the tactile graphics and output to the tactile interface device, it may be difficult for the user to understand the tactile graphics.

Therefore, according to an embodiment of the present invention, a focus area including a part of the graphic display output from the application 3000 is converted into tactile graphic data. That is, only the focus area corresponding to the area focussed by the user or the focus area is converted into tactile graphic data, and the tactile graphic data is displayed on the tactile graphic data so that the user can recognize the current focused area only quickly and conveniently. The error in the input can be eliminated.

Meanwhile, the focus area is determined by at least one of the previous inputs by the user. For example, when the user moves the input coordinates of the tactile interface device with the touch input and focuses the area of the input coordinate by a gesture such as a tap, the focusing area is determined by the inputs of the user .

Meanwhile, the haptic graphic converter 2210 divides the focus area into a plurality of focus area and converts the display elements of each focus area into tactile graphic elements according to a predetermined conversion rule Tactile graphic data.

Fig. 20 shows an example of a predetermined conversion rule for converting such a display element into a tactile graphic element.

For example, when the display element (GUI element) corresponds to 'Button' in FIG. 20, the form displayed on the actual screen of the application 3000 is the GUI of the form shown in the item of 'Visual Design' in FIG. Then, the display element for the 'Button' is converted into a tactile icon according to the conversion rule shown in FIG. 20, and the converted tactile icon is implemented as a tactile graphic in the tactile interface device. On the other hand, detailed information on the tactile icon can be recognized by the user in the form of braille tactile graphics of the tactile interface device or voice by the TTS conversion by means of an input of the user's tactile interface device, preferably a gesture type input have.

19 is a view schematically showing a tactile graphics 4000 according to an embodiment of the present invention. The tactile graphics include a plurality of layers, and in FIG. 19, the plurality of layers include a first layer 4100, a second layer 4200, and a third layer 4300.

The plurality of focus partial areas correspond to the plurality of layers, respectively. That is, the focus area including a part of the graphic display output from the application 3000 is composed of a plurality of focus partial areas, and each of the plurality of focus partial areas corresponds to each of the plurality of layers. Accordingly, the tactile graphics converting unit 2210 converts the graphic display into the tactile graphics by the focus area or the layer.

The first layer 4100 is a screen control area. The first layer 4100 is an icon for indicating a title of a tactile graphic output from the current tactile interface device, a button for switching the screen such as 'previous', 'confirm', 'cancel' And tactile icons of the buttons. That is, the first layer 4100 includes a tactile icon for executing a command to change the screen of the graphic display output from the application 3000. [

The second layer 4200 is a main screen area in which tactile graphics for buttons, menus, folders, graphic information, etc. are provided. The user may perform drawing input, recognize graphic information, recognize GUI elements, or perform user control input through the second layer 4200.

The third layer 4300 is a layer for displaying detailed information on tactile graphic elements output from the application 3000 in braille form. The user can confirm detailed information about the tactile elements by recognizing the braille output to the third layer 4300. As described above, confirmation of such detailed information can also be performed by the voice converted by the TTS converting unit 2220. [

According to the above-described tactile interface device and the electronic device (B) for controlling the same, a program such as 'Paint' corresponding to the WINDOWS ^TM auxiliary program can be used by the visually impaired. The blind person can input a picture including the line through the second layer 4200, recognize the picture drawn by himself / herself in real time, and continuously complete the picture based on the recognized picture information I can go out.

Although the tactile-sensing interface device and the electronic device for controlling the tactile-sensing interface device described above have been described in terms of the tactile-sensing interface device capable of outputting and inputting and the control thereof, the present invention is not limited thereto, But also includes inventions for control methods, apparatus, and computer-readable storage media for tactile interface devices that are only capable of being used.

15 is a diagram schematically illustrating the steps of a method for controlling a tactile interface device according to an embodiment of the present invention.

A method of controlling a tactile interface device electrically connected to an electronic device B including an application 3000 executed internally as shown in Figure 15 and capable of interacting with a user, (S100) for converting an input from the application (3000) into an input to the application (3000); And an output conversion step (S200) for converting an output from the application (3000) to an output of the tactile interface device.

The tactile interface device may provide tactile tactile graphics, and the input conversion step S100 and the output conversion step S200 are preferably performed simultaneously. Specifically, the user recognizes the tactile graphics output by the output conversion step S200, performs the input accordingly, and performs the input conversion step S100 in such a form that the input of the user can be input to the application 3000. [ . ≪ / RTI >

Here, the electronic device B includes a smart phone, a tablet, a personal computer (PC), a mobile phone, a video phone, an electronic book An e-book reader, a desktop PC, a laptop PC, a netbook PC, a personal digital assistant (PDA) A portable multimedia player (PMP), an MP3 player, a mobile medical device, a camera, a wearable device (for example, a portable multimedia player (PMP) A head-mounted device (HMD), an electronic garment, an electronic bracelet, an electronic necklace, an electronic app apparel, an electronic tattoo, or a smart watch (smart watch) or the like, and preferably A PC flow. In the electronic device (B) comprises as at least one processor and a memory, and selectively fixed to the display device such as a monitor, or may be provided with its own display module.

Alternatively, the electronic device B may be in combination with the tactile interface device. In this case, the tactile interface device and the electronic device B controlling the tactile interface device may be recognized as a single device. In the case of such a combined device, the electronic device B and the tactile interface device may be in the form of sharing a memory with the processor.

The application (e.g. Browser in WINDOWS ^TM) application to perform a range of functions provided by OS in, for applications (e.g., for performing a function such as provided by an auxiliary in the OS, Paint in WINDOWS ^TM ), An application that is installed separately from the OS and performs its own functions (for example, MS WORD ^TM ), an application that performs its own functions that run on the Web, an application on a smartphone, or an application . That is, the application collectively refers to programs having executable instructions.

16 is a view schematically showing the processes of the input conversion step S100 according to an embodiment of the present invention.

The input conversion step S100 may include an input information receiving step (S110) of receiving input information input from the tactile interface device; An input information category distinguishing step (S120) for distinguishing the category of the input information; An input information converting step (S130) of converting the input information into a form that can be input to the application (3000) according to the category; And an input information transmitting step (S140) for transmitting the converted input information to the application.

Meanwhile, the category includes at least one of a key input and a touch input.

The key input is an input by a braille keyboard that converts character input signals to a computer by converting braille characters commonly used by the visually impaired to general characters. The Braille keyboard transmits the Braille to the user terminal connected to the tactile interface device or the tactile interface device as well as the conventional braille keyboard. Braille keyboard consists of 1 ~ 6 points, backspace, space, and enter button. Braille consists of several dots to form one letter, so the braille keyboard can transmit information of pressed buttons at the same time.

The touch input refers to an input form including all of input by non-contact type optical based sensing shown in FIGS. 7 and 8, input by touch recognition by capacitance change generally used in a tablet PC, smart phone, and the like. In other words, the touch input includes not only a key input type but also an input type in which a user directly inputs the touch input.

If the category of the input information is a touch input, the input information conversion step (S130) determines whether the touch input is matched with any one of a plurality of predefined gestures. If the category is the touch input, And converts the input information into a command that can be input to the application 3000 corresponding to the matched gesture.

Here, an example of the predetermined gesture is shown in Table 1 described above.

The interaction between the tactile interface device and the electronic device is primarily performed through the gesture recognition function of the touch input. For the blind and intuitional computer operation, various touch operations as shown in Table 1 are defined, It is preferable to transfer it as an input signal to the electronic device or the application 3000 of the electronic device.

For example, a single tap gesture outputs the type of GUI (Graphic User Interface) element and corresponding text to the tactile icon at the touched position in braille and voice. For example, a blind person recognizes a tactile icon recognized by a tactile graphics through a swipe operation, and performs a single tap gesture on the tactile icon to move the focus to the corresponding GUI element. In addition, the tab gesture may output the detailed information of the GUI element to be selected, for example, text information in braille to one of the layers displaying tactile graphics of the tactile interface device, and also output it as TTS-based voice.

Long tap, long tap after tap, panning after tap, and drawing circle after tap corresponding to the remaining gestures in Table 1 are performed once, followed by tap, long tap, panning, and drawing the circle gestures . This is so defined that the finger swipe motion for recognizing the tactile graphics of a visually impaired person and the defined gesture are confused and not recognized. The double tap gesture has the same function as the double left click of the mouse that executes the default action of the GUI element by touching twice in the same position quickly. Long tap after tap The gesture is a tap that is touched at the same position and touched continuously for a certain amount of time, and has the same function as a right click of the mouse. Panning after tap The gesture is an operation to perform a panning operation by touching again at the same position after performing a tap once, and has a function of scrolling up / down / left / right according to the panning direction. Drawing circle after tap A gesture is a motion that draws a circle by quickly touching again at the same position after performing a tap once. It has a screen enlargement function in the clockwise direction and a screen reduction function in the counterclockwise direction.

If the category of the input information is a touch input, the input information conversion step (S130) determines whether or not the touch input corresponds to the input of the input coordinates. If the touch input corresponds to the input coordinates, And converts the input coordinates input from the device into the input coordinates in the application 3000.

The input information conversion step (S130) for converting the coordinates as described above is for converting coordinates or input coordinates in the tactile interface device into coordinates in the application 3000.

When the electronic device is a PC, tactile icons are implemented in the tactile graphics outputted by reconfiguring the GUI elements of the monitor display screen of the PC. Since the tactile icon is not matched with the monitor display screen in a spatial 1: 1 manner, when the input coordinates at which the touch input occurs in the tactile interface device are used as they are, It can not be discriminated by the application 3000. The input information conversion step (S130) for the coordinates substantially corresponds to the coordinate mapping function. That is, a mapping table is configured to map the tactile icon object currently being output as a tactile graphics and the GUI coordinates output to the monitor display screen when driving the actual application 3000, thereby mapping the recognized touch coordinates.

The tactile graphics displayed on the tactile interface device and the display screen when the application 3000 is generally executed may differ in resolution, configuration, and position. In this case, 3000, the input coordinates in the tactile interface must be converted or mapped to the input coordinates in the application 3000, and this is performed by the input information conversion step S 130. In this way, the user can use the function of the application 3000 at a general user level without reducing or simplifying the function of the application 3000.

For example, in order to execute a command 'A' in the application 3000 (for example, when the user opens the My Documents folder on the Windows desktop), the tactile interface device When performing Double Tab (see Table 1 above), the coordinate mapping unit 2130 maps the coordinates of the point in the tactile interface device to the coordinates of the icon 'A' of the connected application 3000, The 'A' command in the application 3000 can be executed.

As described above, in the input information conversion step (S130), a touch gesture recognition function for recognizing a gesture with respect to a touch input of the tactile interface device, a coordinate mapping function for indicating a GUI element corresponding to the touch coordinates input from the touch recognition device, Also, it is possible to perform a reverse grayscale function for converting the braille information input from the braille keyboard into general characters.

17 is a view schematically showing the processes of the output conversion step S200 according to an embodiment of the present invention. The output conversion step S200 is a step of converting a focus area including a part of the graphic display output from the application 3000 into tactile graphic data.

The output conversion step (S200)

Receiving output information of the graphic display from the application 3000 (S210);

Determining whether the graphic display displayed by the application 3000 has been changed (S220);

A tactile graphic conversion step (S230) of converting the graphic display displayed by the application (3000) into tactile graphic data; And

And transmitting the tactile graphic data to the tactile interface device (S240).

18 is a view schematically showing processes of a tactile graphic conversion step S230 according to an embodiment of the present invention.

When the actual application 3000 is driven, when all the GUI elements output to the monitor display screen are converted into tactile graphics and output to the tactile interface device, it may be difficult for the user to understand the tactile graphics.

Therefore, according to an embodiment of the present invention, the tactile graphic conversion step S230 may convert the focus area including a part of the graphic display output from the application 3000 into tactile graphic data. That is, only the focus area corresponding to the area focussed by the user or the focus area is converted into tactile graphic data, and the tactile graphic data is displayed on the tactile graphic data so that the user can recognize the current focused area only quickly and conveniently. And errors in the input can be eliminated.

Meanwhile, the focus area is determined by at least one of the previous inputs by the user. For example, when the user moves the input coordinates of the tactile interface device with the touch input and focuses the area of the input coordinate by a gesture such as a tap, the focusing area is determined by the inputs of the user .

According to an exemplary embodiment of the present invention, the tactile graphic conversion step S230 may include dividing the focus area into a plurality of focus part areas, and displaying the display elements of the respective focus part areas according to a predetermined conversion rule as a tactile graphic element Thereby generating tactile graphics data.

18, the tactile graphic conversion step S230 for converting the graphic display displayed by the application 3000 into the tactile graphic data includes a step of displaying a part of the graphic display outputted from the application 3000, A focus area determination step (S231) of determining an area;

A focus partial area dividing step (S232) of dividing the focus area into a plurality of focus partial areas;

A tactile graphic element conversion step (S233) of converting a display element of each focus partial area into a tactile graphic element according to a predetermined conversion rule; And

And tactile graphic data generation step S234 for generating tactile graphic data including the tactile graphic element.

Fig. 20 shows an example of a predetermined conversion rule for converting such a display element into a tactile graphic element.

For example, when the display element (GUI element) corresponds to 'Button' in FIG. 20, the form displayed on the actual screen of the application 3000 is the GUI of the form shown in the item of 'Visual Design' in FIG. Then, the display element for the 'Button' is converted into a tactile icon according to the conversion rule shown in FIG. 20, and the converted tactile icon is implemented as a tactile graphic in the tactile interface device. On the other hand, detailed information on the tactile icon can be recognized by the user in the form of braille tactile graphics of the tactile interface device or voice by the TTS conversion by means of an input of the user's tactile interface device, preferably a gesture type input have.

19 is a view schematically showing a tactile graphics according to an embodiment of the present invention. The tactile graphics include a plurality of layers, and in FIG. 19, the plurality of layers include a first layer 4100, a second layer 4200, and a third layer 4300.

The plurality of focus partial areas correspond to the plurality of layers, respectively. That is, the focus area including a part of the graphic display output from the application 3000 is composed of a plurality of focus partial areas, and each of the plurality of focus partial areas corresponds to each of the plurality of layers. Accordingly, the tactile graphics converting unit 2210 converts the graphic display into the tactile graphics by the focus area or the layer.

The first layer 4100 is a screen control area. The first layer 4100 is an icon for indicating a title of a tactile graphic output from the current tactile interface device, a button for switching the screen such as 'previous', 'confirm', 'cancel' And tactile icons of the buttons. That is, the first layer 4100 includes a tactile icon for executing a command to change the screen of the graphic display output from the application 3000. [

The second layer 4200 is a main screen area in which tactile graphics for buttons, menus, folders, graphic information, etc. are provided. The user may perform drawing input, recognize graphic information, recognize GUI elements, or perform user control input through the second layer 4200.

The third layer 4300 is a layer for displaying detailed information on tactile graphic elements output from the application 3000 in braille form. The user can confirm detailed information about the tactile elements by recognizing the braille output to the third layer 4300. As described above, confirmation of such detailed information can also be performed by the voice converted by the TTS converting unit 2220. [

According to the method of controlling the tactile interface device as described above, a program such as 'Paint' corresponding to the auxiliary program of WINDOWS ^TM can be used by the visually impaired. The blind person can input a picture including the line through the second layer 4200, recognize the picture drawn by himself / herself in real time, and continuously complete the picture based on the recognized picture information I can go out.

In the meantime, the above-described method of controlling the tactile interface device has been described in terms of the tactile interface device capable of outputting and inputting and the control thereof. However, the present invention is not limited thereto. And a control method for the tactile interface device.

Hereinafter, embodiments of the graphic display output from the application 3000 that is converted by the method of controlling the tactile display device according to the present invention will be described. In the following description, it is assumed that a predetermined conversion rule for converting a display element of the graphic display shown in Fig. 20 into a tactile graphic element is applied.

21 is a diagram showing a first example of a display screen (graphic display) of the electronic device B;

21, the focus area corresponds to the window in which the application is executed, and the focus partial area includes the first focus partial area including (1), (2), (3), and (4) And a second focus partial area.

The above (1) corresponds to the title of this application, (2) corresponds to the menu bar of this application, (3) corresponds to the 'minimize' button of this application, Quot; close " button, and (5) corresponds to a window for displaying objects displayed in the application.

FIG. 22 is a diagram showing a tactile graphics outputted by converting a first example of the display screen shown in FIG. 21 into a tactile graphics according to an embodiment of the present invention.

The first focus partial area including the above-mentioned (1), (2), (3) and (4) is converted or mapped to the upper layer (first layer 4100) (Second layer 4200) of tactile graphics is converted or mapped. As described above, the conversion rule at the time of mapping is in accordance with the rule shown in FIG.

On the other hand, the lower layer of the tactile graphic corresponds to a layer in which text information of the tactile icon referred to by the user in the first layer 4100 and the second layer 4200 is expressed in braille.

For example, the area (1) corresponds to a title, which is converted into a predetermined tactile icon by the conversion rule shown in FIG. 20, and is implemented in the first layer 4100 of the tactile graphics. On the contrary, when the user takes a gesture such as a tap on the tactile icon corresponding to the area (1), the third layer 4300 outputs 'Haptic Editor' corresponding to the detailed information of the title in braille form.

23 is a diagram showing a second example of a display screen (graphic display) of the electronic device B;

23 is a graphical display when a menu bar corresponding to the area (2) is selected in the application shown in FIG. 21 and 'Page Control' is selected or selected.

In this case, the focus area corresponds to an area called " Focused area " in FIG. 23, and the focus part area includes a first focus partial area including (1) and a second focus partial area including (2) And can be divided into a focus partial area.

The above (1) corresponds to the title selected in the menu bar, and the above (2) to (6) correspond to the submenu that can be selected within the title.

FIG. 24 is a diagram showing a tactile graphics output by converting a second example of the display screen shown in FIG. 23 into a tactile graphics according to an embodiment of the present invention.

The first focus partial area including the above (1) and (7) is transformed or mapped to the upper layer (first layer 4100) of the tactile graphics, and the second focus partial area including the second (2) The focus subarea is transformed or mapped to the interruption layer of the tactile graphics (second layer 4200). As described above, the conversion rule at the time of mapping is in accordance with the rule shown in FIG. Meanwhile, (7) is a tactile icon for the 'backward' button, which is not actually included in the graphic display of the application, but corresponds to an icon generated by itself in order to extend the user's convenience in the control method according to the present invention.

On the other hand, the lower layer of the tactile graphic corresponds to a layer in which text information of the tactile icon referred to by the user in the first layer 4100 and the second layer 4200 is expressed in braille.

For example, the area (1) corresponds to a title, which is converted into a predetermined tactile icon by the conversion rule shown in FIG. 20, and is implemented in the first layer 4100 of the tactile graphics. On the other hand, if the user takes a gesture such as a tap on the tactile icon corresponding to the area (1), the third layer 4300 outputs 'Page Control' corresponding to the detailed information of the title in braille form.

FIG. 25 is a view schematically showing a process of drawing a picture by a blind person with a tactile interface device according to an embodiment of the present invention.

According to the tactile interface device and the electronic device B controlling the same as in the present invention, a program such as 'Paint' corresponding to the auxiliary program of WINDOWS ^TM can be used by the visually impaired. That is, when the application is an application capable of performing a function of inputting, editing, and storing graphics, a blind person can utilize graphic input and related functions executed in a computing device which has not been utilized by existing blind persons . Specifically, a blind person can input a picture including a line through the second layer 4200, recognize a picture drawn by himself / herself using a tactile sense in real time, Can be completed. That is, according to one embodiment of the present invention, a blind person can easily perform graphic input, editing, and storage with a computer.

The method, apparatus, and computer-readable medium for controlling a tactile interface device in accordance with the present invention can exert an effect of visually impaired persons intuitively recognizing tactile display items in real time.

In addition, the method, apparatus and computer-readable medium for controlling the tactile interface device according to the present invention allow the visually impaired to recognize the information tactually and at the same time instantly input based on the perceived information .

In addition, the method, apparatus and computer-readable medium for controlling the tactile interface device according to the present invention can be applied to a tactile display capable of recognizing tactile and gesture- So that it is possible to promptly and intuitively input information to the user, so that the computer can be effectively used in a manner very similar to that of the general public.

Further, the method, apparatus, and computer-readable medium for controlling the tactile interface device according to the present invention can exhibit the effect of being easily expandable to use various functions of the OS and various application software.

In addition, the method, apparatus and computer-readable medium for controlling the tactile interface device according to the present invention can provide a visually impaired person with a speedy and convenient operation speed that is remarkably improved in comparison with a device such as a screen reader of the related art .

In addition, the method, apparatus, and computer-readable medium for controlling the tactile interface device according to the present invention can exert the effect that the visually impaired persons can conveniently perform graphic related tasks that are difficult to perform with a screen reader.

Further, according to a method, an apparatus and a computer-readable medium for controlling a tactile interface device according to the present invention, WINDOWS ^TM Programs such as 'Paint', which is an auxiliary program, are also available for the visually impaired. The blind can input a picture including the line through the second layer, recognize the picture drawn by himself / herself using the tactile sense in real time, and continue to complete the picture based on the recognized picture information .

Further, the method, apparatus and computer-readable medium for controlling the tactile interface device according to the present invention significantly improve the ability of the visually impaired to utilize the computer, and to expand the market for computer hardware and software for the visually impaired .

In one or more exemplary implementations, the functions presented herein may be implemented in hardware, software, firmware, or a combination thereof. When implemented in software, the functions may be stored on or transmitted via one or more instructions or code on a computer readable medium. Computer-readable media includes computer storage media and communication media including any medium for facilitating transfer of a computer program from one place to another. The storage medium may be any general purpose computer or any available medium that can be accessed by a special purpose computer. By way of example, and not limitation, such computer-readable media can comprise any form of computer readable medium, such as RAM, ROM, EEPROM, CD-ROM or other optical disk storage media, magnetic disk storage media or other magnetic storage devices, But not limited to, a general purpose computer, a special purpose computer, a general purpose processor, or any other medium that can be accessed by a particular processor.

In addition, any connection means may be considered as a computer-readable medium. For example, if the software is transmitted from a web site, server, or other remote source over wireless technologies such as coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or infrared radio, and microwave, Wireless technologies such as cable, fiber optic cable, twisted pair, DSL, or infrared radio, and microwave may be included within the definition of such medium. The disks and discs used herein may be compact discs (CDs), laser discs, optical discs, DVDs, floppy disks,

This disc contains discs where discs reproduce data magnetically, while discs reproduce data optically through a laser. The combinations may also be included within the scope of computer readable media.

Those of ordinary skill in the art will appreciate that the various illustrative elements, components, logical blocks, modules, and algorithm steps described above may be implemented as electronic hardware, computer software, or combinations thereof. In order to clarify the interchangeability of hardware and software, various illustrative components, blocks, modules and steps have been described in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement these functions in varying ways for each particular application, but such implementation decisions are not necessarily outside the scope of the invention.

The various illustrative logical blocks and modules described herein may be implemented or performed with a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, Discrete hardware components, or any combination designed to implement the functions described herein. A general purpose processor may be a microprocessor; In an alternative embodiment, such a processor may be an existing processor, controller, microcontroller, or state machine. A processor may be implemented as a combination of computing devices, such as, for example, a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or a combination of such configurations.

For a hardware implementation, the various illustrative logic, logic blocks, and modules of processing units described in connection with the aspects disclosed herein may be implemented with one or more application specific integrated circuits (ASICs), digital signal processors (DSPs) Such as digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), discrete gate or transistor logic, discrete hardware components, general purpose processors, Controllers, microprocessors, other electronic units designed to perform the functions described herein, or a combination thereof. The general purpose processor may be a microprocessor, but, in the alternative, it may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices (e.g., a DSP and a microprocessor, a plurality of microprocessors, a combination of one or more microprocessors in conjunction with a DSP core, or any other suitable configuration). Additionally, at least one processor may include one or more modules capable of implementing one or more of the steps and / or operations described herein.

In addition, various aspects or features described herein may be implemented as a method, apparatus, or article of manufacture using standard programming and / or engineering techniques. Moreover, steps and / or operations of a method or algorithm described in connection with the aspects disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. Additionally, in some aspects, steps or acts of a method or algorithm may be present as a machine-readable medium, or as a combination of at least one or any combination of codes or instructions on a computer-readable medium, It can be integrated into computer program stuff. The term article of manufacture as used herein is intended to encompass a computer program accessible from any suitable computer-readable device or medium.

The description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features presented herein.

Claims (8)

CLAIMS What is claimed is: 1. A method of controlling a tactile interface device that is electrically connected to an electronic device including an application running internally and capable of interacting with a user,
An input conversion step of converting an input from the tactile interface device into an input to the application; And
And an output conversion step of converting an output from the application into an output of the tactile interface device,
Wherein the tactile interface device is capable of providing tactile tactile graphics.
The method of claim 2,
Wherein the input conversion step comprises:
An input information receiving step of receiving input information inputted from a tactile interface device; And
And an input information conversion step of converting the input information into a form that can be input to the application.
Claim 2
Wherein when the input information is a touch input, the input information conversion step determines whether the touch input matches a predetermined plurality of gestures,
In response to a match, converting the input information into a command that can be input to the application corresponding to the matched gesture.
In claim 2,
If the category of the input information is a touch input, the input information conversion step determines whether the touch input corresponds to the input of the input coordinates,
And converts the input coordinates input from the tactile interface device to input coordinates in the application when the touch input corresponds to the input coordinate.
The method of claim 2,
Wherein the output conversion step includes a tactile graphic conversion step of converting a focus area including a part of the graphic display output from the application into tactile graphic data,
Wherein the tactile graphic conversion step comprises:
A focus area determination step of determining a focus area including a part of the graphic display output from the application;
A focus partial area dividing step of dividing the focus area into a plurality of focus partial areas;
A tactile graphic element conversion step of converting a display element of each focus partial area into a tactile graphic element according to a predetermined conversion rule; And
And a tactile graphic data generation step of generating tactile graphic data including the tactile graphic element,
Wherein the focus area is determined by at least one of a previous input of a user.
The method of claim 2,
Wherein the tactile graphic includes a plurality of layers,
Wherein the plurality of focus partial areas correspond to the plurality of layers,
Wherein one of the plurality of layers is a layer that displays text information of a tactile graphic element referred to as a touch input by the user in braille form.
The method of claim 2,
Wherein the tactile graphic includes a plurality of layers,
Wherein one of the plurality of layers displays a tactile icon for executing a command to change a screen of the graphic display output from the application.
22. A computer-readable medium,
The computer-readable medium stores instructions that cause a computer to perform the following steps:
An input conversion step of converting an input from a tactile interface device capable of interacting with a user into an input to an application executed in the electronic device; And
And an output conversion step of converting an output from the application into an output of the tactile interface device,
Wherein the tactile interface device is capable of providing tactile tactile graphics.

Images