KR102078363B1 - Method, Device, and Non-transitory Computer-Readable Medium for Providing Image Viewer Function By Tactile Interface Device - Google Patents

Abstract

The present invention relates to a method, a device and a non-transitory computer readable medium for providing an image viewer function through a tactile interface device, and more particularly, to a method, a device and a non-transitory computer readable medium for providing an image viewer function through a tactile interface device that allows the visually impaired to intuitively detect an image by touching, enlarging, reducing and moving the image by using, interacting with and controlling a tactile interface device corresponding to a multi-array tactile cell based smart braille device to increase the efficiency of use of smart devices such as smartphone, smart pad, etc. or the smart braille device and improve the level of informatization.

Description

Method, Device, and Non-transitory Computer-Readable Medium for Providing Image Viewer Function By Tactile Interface Device that provides an image viewer function through a tactile interface device}

The present invention relates to a method, a device for providing an image viewer function through a tactile interface device, and a non-transitory computer-readable medium, and more specifically, a smart device (smartphone, smart pad, etc.) or smart braille for the visually impaired. In order to increase the efficiency of use of the device and increase the level of informatization, the visually impaired person can intuitively tactilely sense the image by enabling the intuitive use, interaction, and control of the tactile interface device corresponding to the multi-array tactile cell-based smart braille device. A method, apparatus, and a non-transitory computer-readable medium that provides an image viewer function through a tactile interface device that enables detection and enlargement, reduction, and movement of an image.

In the information society, the acquisition and use of information using computers is essential, and this trend is the same for the visually impaired as well as the general public.

In order to improve the level of informatization for the visually impaired, they must also be able to acquire and use information at a level similar to that of normal people, which is important in that it makes the daily life of the visually impaired more convenient and enables users to acquire and utilize information on their own. Do. In addition, it can be said that this is very important in that it can ultimately improve the well-being by providing various educational opportunities for the visually impaired and expanding opportunities for social advancement and participation.

However, the current general method of using a computer mainly uses a method of recognizing visual information output through a monitor with an eye and inputting information using an input tool such as a keyboard, mouse, or touch pad. Therefore, there is a limitation in that a visually impaired person who cannot use vision can recognize the output information of the computer at the same level as the general public and interact with such as instant information input. As a result, the visual impediment significantly lowers the efficiency in using the computer, greatly depriving the computer of the opportunity to acquire and utilize information.

Various techniques have been developed to recognize visual information using hearing and tactile senses and interact with the computer through the visually impaired to solve difficulties in using the computer. As a representative technology, there is a screen reader that assists a visually impaired person to use a computer through hearing. This is a device or software that helps you use the computer by outputting the information displayed on the computer screen and the keyboard information that you have input by voice.

However, since the screen reader searches for Graphic User Interface (GUI) elements of the output screen with only linearized information in one line without two-dimensional spatial location information on the output screen, it is difficult to recognize screen output information, especially included in the output screen The more information, the greater the difficulty. In addition, for various graphic information such as pictures and diagrams in addition to text and GUI elements, the screen reader provides only text-based simple descriptive descriptions by voice, so that visually impaired people can understand and interact with graphic information. You will have great difficulty.

Another related technology is a braille information terminal that transmits text information tactilely through a braille cell. This is a method that is used as an independent device by providing some functions of a computer useful for the visually impaired, and a method that is used as a screen output auxiliary device that outputs text information of a computer screen analyzed by a screen reader in Braille. In both ways, rather than acting as an interface for efficient interaction with a computer, it acts as an alternative device that performs only a few limited functions of the computer as a substitute for a computer, or as an output auxiliary device that outputs text information in Braille. do. In particular, a braille information terminal specialized in braille output has a problem that it cannot express graphic information like a screen reader.

Korean Patent Publication No. 10-2012-0063982

An object of the present invention is to intuitively use a tactile interface device corresponding to a multi-array tactile cell-based smart braille device to increase the efficiency of use of smart devices (smartphones, smart pads, etc.) or smart braille devices for the visually impaired and improve the level of information How to provide the image viewer function through the tactile interface device that enables the visually impaired to sense the image tactilely by using, interacting, and controlling it. , Devices, and non-transitory computer-readable media.

In order to solve the above problems, the present invention is implemented as a computing device including a processor, and is a method of providing an image viewer function through a tactile interface device that is connected to the computing device and can interact with a user. When the image viewer module is executed in the computing device, one of the main screen step, the image selection step, and the image output step is performed, and when the image viewer module is executed, the screen according to the corresponding step is displayed on the computing device. , A screen displayed on the computing device is converted into an output signal to the tactile interface device, so that a tactile display corresponding to the screen displayed on the computing device can be displayed on the tactile interface device, or input from the computing device, or Main screen step according to the input from the tactile interface device, Provides a way to provide any of the image viewer function that details the process steps are performed during image selection and image output stage.

In the present invention, the tactile display displayed by the main screen step and the image selection step includes: a cursor area tactilely displaying information on a current cursor position among a plurality of cursor positions; A content area tactilely displaying information corresponding to the position of each cursor; And a page area tactilely displaying information on the current page. It may include.

In the present invention, the image selection step comprises: an image database list display step of displaying a list of one or more image databases in which images are stored; An image database selection step of selecting an image database from which an image is to be loaded by a user input; An image category display step of displaying a category of images stored in the image database; An image category selection step of selecting a category by user input; An image search step of searching for an image by user input; An image list display step of displaying a list of printable images based on at least one of the selected image database, the selected category, and the image search results; And a display image selection step of selecting an image to be output by a user input. It may include.

In the present invention, the image output step, the image discrimination step of determining the type of the image based on the information of the image to be output; An image simplification step of simplifying the image to be output by the tactile interface device based on the determined type of the image; An image display step of outputting the simplified image through the computing device and the tactile interface device; And an output image adjustment step of enlarging, reducing or moving the image output through the tactile interface device to output the image. It may include.

In the present invention, the image simplification step comprises: a grayscale conversion step of converting the image to grayscale to generate a grayscale image; And an image binarization step of binarizing the grayscale image to generate a binarized image. It may include.

In the present invention, the image display step comprises: an output area determining step of setting an area to be output through the tactile interface device among the images as an output area; A first display information generating step of generating first display information including information of the image to be displayed on a computing device based on the image; And a second display information generating step of generating second display information including image information of an output area of a simplified image to be tactilely displayed in a tactile interface device based on the image. It may include.

In the present invention, the output image adjustment step includes: an enlargement / reduction of an output image to enlarge or reduce the image tactilely displayed by the tactile interface device and output the image; And an output image moving step of moving and outputting the position of the image tactilely displayed by the tactile interface device. It may include.

In the present invention, the tactile display displayed by the image output step includes: a content area tactilely displaying image information of the output area of the simplified image; And a location information area tactilely displaying information on the output area. It may include.

In the present invention, the first display information further includes information of the output area, and the image and the output area may be displayed on the computing device displayed by the image output step.

In the present invention, in the second display information generation step, the information of the output region of the binarized image is mapped to the coordinates of the pin in a plurality of pins of the tactile interface device to determine the rise or fall of each pin. The second display information can be generated.

In the present invention, in the second display information generation step, when mapping the information of the output region of the binarized image to the coordinates of the pin, when the ratio of black in the region matching the coordinates of the pin is 50% or more, the pin Raise the pin completely, and when the proportion of black in the area matching the coordinates of the pin is less than 20%, completely lower the pin, and when the proportion of black in the area matching the coordinates of the pin is 20% or more and less than 50%, remove the pin. The second display information may be generated to be positioned in the middle.

In the present invention, the second display information generation step is a step of rising or falling each of the pins by mapping the information of the output region of the grayscale image to the coordinates of the pins in a plurality of pins of the tactile interface device. By determining, the second display information may be generated.

In order to solve the above problems, the present invention is implemented as a computing device including a processor, and is an apparatus that provides an image viewer function through a tactile interface device that is connected to the computing device and can interact with a user. The computing device includes an image viewer module; Including, the image viewer module, an image selection unit for selecting an image to be output by the user's input; And an image output unit outputting the selected image through the computing device and the tactile interface device. Including, The image output unit, Image discrimination step of determining the type of the image based on the information of the image to be output; An image simplification step of simplifying the image to be output by the tactile interface device based on the determined type of the image; An image display step of outputting the simplified image through the computing device and the tactile interface device; And an output image adjustment step of enlarging, reducing or moving the image output through the tactile interface device to output the image. Provides a device that provides an image viewer function, to perform the.

In order to solve the above problems, the present invention is a non-transitory computer-readable medium for providing an image viewer function to a visually impaired person by controlling a tactile interface device connected to a computing device, wherein the computer-readable medium is computing Storing commands to cause the device to perform the following steps, the steps comprising: a main screen step of outputting an image viewer main screen that provides an image viewer function to a visually impaired person; An image selection step of selecting an image to be output by the user's input; And an image output step of outputting the selected image through the computing device and the tactile interface device. Including, The image output step, The image discrimination step of determining the type of the image based on the information of the image to be output; An image simplification step of simplifying the image to be output by the tactile interface device based on the determined type of the image; An image display step of outputting the simplified image through the computing device and the tactile interface device; And an output image adjustment step of enlarging, reducing or moving the image output through the tactile interface device to output the image. It provides a computer-readable medium comprising a.

According to an embodiment of the present invention, a visually impaired person uses a braille device or a tactile interface device to display various contents (books, education, multimedia, internet, games, etc.) in a level similar to the general public in everyday life such as education, culture, and leisure. Tactile User Interface for multi-array tactile cell-based smart braille devices and the main operating system based braille devices such as Android for application interlocking / controlling / utilization or tactile interface device operating system Can be exercised.

According to an embodiment of the present invention, the visual information (visually perceptible information) provided by the smart device to the visually impaired is converted into tactile information (tactilely perceptible information) in real time to multi-array braille cells. It can have the effect of providing a smart braille device operating system that can output through the tactile display, manage various input information of the user, and control and manage various application software operating on the smart device.

According to an embodiment of the present invention, a multi-array tactile cell-based smart braille device input unit (braille keyboard / direction key / function key, etc.) controls the operation of the smart braille device's operating system (Android OS) and applications, and performs general smart By converting GUI (Graphic User Interface) information output from the device to TUI (Tactile User Interface) data for the visually impaired and providing it as tactile and braille information, the visually impaired is a general smart device or smart braille device at a level similar to the general public. It can be easily controlled to exert the effect of using various contents.

According to an embodiment of the present invention, the effect of allowing the visually impaired to interpret the image file at a level similar to the general public through the tactile sense of the stored image based on the multi-array tactile cell-based smart braille device and the visually impaired TUI. Can be exercised.

According to an embodiment of the present invention, an image can be output through a smart braille device based on a multi-array tactile cell by simplifying an image through grayscale and binarization.

According to an embodiment of the present invention, it is possible to enlarge, reduce, and move an image output through a multi-array tactile cell-based smart braille device, thereby exerting the effect of allowing the visually impaired to interpret the image file in detail.

According to an embodiment of the present invention, when the output image is enlarged or moved, it is possible to visually impair the interpretation of the image file by allowing the visually impaired user to check the location of the location in the entire image. Can be effective.

1 is a view schematically showing an entire system including a tactile interface device and a computing device according to an embodiment of the present invention.
2 is a diagram schematically showing an internal configuration of a computing device according to an embodiment of the present invention.
3 is a diagram schematically showing an operating environment of a braille OS unit according to an embodiment of the present invention.
4 is a diagram schematically showing the internal configuration of braille OS according to an embodiment of the present invention.
5 is a view schematically showing the internal configuration of a TUI module according to an embodiment of the present invention.
6 schematically illustrates a tactile display screen in a tactile interface device according to an embodiment of the present invention.
7 schematically illustrates a process of a method for controlling a tactile interface device according to an embodiment of the present invention.
8 schematically shows a detailed process of a trigger step according to an embodiment of the present invention.
9 schematically illustrates a display screen in a computing device and a tactile display in the tactile interface device according to an embodiment of the present invention.
10 schematically illustrates a region configuration of a tactile display in a tactile interface device according to an embodiment of the present invention.
11 schematically illustrates an input / output process of a tactile interface device according to an embodiment of the present invention.
12 schematically illustrates a process of an output signal generation step according to an embodiment of the present invention.
13 schematically shows a process of a cursor region output signal generation step according to an embodiment of the present invention.
14 schematically shows an example of a form of a cursor area notation according to an embodiment of the present invention.
15 schematically shows some examples of cursor region notation according to an embodiment of the present invention.
16 schematically illustrates a process of a buffer output signal generation step according to an embodiment of the present invention.
17 schematically shows an example of a TTS step according to an embodiment of the present invention.
18 schematically shows an example of an event step according to an embodiment of the present invention.
19 shows a screen for controlling a tactile interface device according to an embodiment of the present invention.
20 illustrates a screen for controlling a tactile interface device according to an embodiment of the present invention.
21 shows a screen for controlling a tactile interface device according to an embodiment of the present invention.
22 schematically shows the configuration of an image viewer module and a tactile interface device according to an embodiment of the present invention.
23 schematically shows the internal configuration of an image viewer module according to an embodiment of the present invention.
24 schematically shows detailed steps of the operation of the image selection unit according to an embodiment of the present invention.
25 illustrates a screen of an image database list display step and a tactile interface device in which TUI conversion is performed on the image database list display step according to an embodiment of the present invention.
26 shows a screen of an image category display step and a tactile interface device in which TUI conversion is performed on the image category display step according to an embodiment of the present invention.
27 illustrates a screen of an image list display step and a tactile interface device in which TUI conversion is performed on the image list display step according to an embodiment of the present invention.
28 schematically shows detailed steps of an image output step according to an embodiment of the present invention.
29 schematically shows detailed steps of an image simplification step and an image display step according to an embodiment of the present invention.
30 schematically shows detailed steps of an output image adjustment step according to an embodiment of the present invention.
31 illustrates a screen of an output image enlargement / reduction step and a tactile interface device in which TUI conversion is performed on the output image enlargement / reduction step according to an embodiment of the present invention.
32 illustrates a screen of a tactile interface device in which an output image moving step and a TUI conversion is performed on the output image moving step according to an embodiment of the present invention.
33 shows a screen of a tactile interface device in which a TUI conversion is performed on the output image adjustment step and the output image adjustment step according to an embodiment of the present invention.
34 illustrates a screen of an image output step and a tactile interface device in which TUI conversion is performed on the image output step according to an embodiment of the present invention.
35 illustrates a screen of an image output step according to an embodiment of the present invention and a tactile interface device in which TUI conversion is performed on the image output step.
36 is a diagram illustrating an example of an internal configuration of a computing device according to an embodiment of the present invention.

In the following, various embodiments and / or aspects are now disclosed with reference to the drawings. In the following description, for purposes of explanation, a number of specific details are disclosed to aid the overall understanding of one or more aspects. However, it will also be appreciated by those of ordinary skill in the art of the present invention that this aspect (s) can be practiced without these specific details. The following description and the annexed drawings set forth in detail certain illustrative aspects of the one or more aspects. However, these aspects are exemplary and some of the various methods in the principles of the various aspects may be used, and the descriptions described are intended to include all such aspects and their equivalents.

In addition, various aspects and features will be presented by a system that may include multiple devices, components and / or modules, and the like. Various systems may also include additional devices, components and / or modules, and / or may not include all of the devices, components, modules, etc. discussed in connection with the drawings. It must be understood and recognized.

As used herein, "an embodiment", "yes", "a good", "an example", etc., may not be construed as any aspect or design described being better or more advantageous than another aspect or designs. . The terms '~ unit', 'component', 'module', 'system', and 'interface' used in the following generally refer to a computer-related entity, for example, hardware, hardware It can mean a combination of software and software.

Also, the terms “comprises” and / or “comprising” mean that the feature and / or component is present, but excludes the presence or addition of one or more other features, components, and / or groups thereof. It should be understood as not.

Further, terms including ordinal numbers such as first and second may be used to describe various components, but the components are not limited by the terms. The terms are used only for the purpose of distinguishing one component from other components. For example, the first component may be referred to as a second component without departing from the scope of the present invention, and similarly, the second component may be referred to as a first component. The term and / or includes a combination of a plurality of related described items or any one of a plurality of related described items.

In addition, in the embodiments of the present invention, unless otherwise defined, all terms used herein, including technical or scientific terms, are generally understood by those skilled in the art to which the present invention pertains. It has the same meaning as that. Terms, such as those defined in a commonly used dictionary, should be interpreted as having meanings consistent with meanings in the context of related technologies, and are ideally or excessively formal meanings unless explicitly defined in the embodiments of the present invention. Is not interpreted as

1. As a virtual operating system that controls tactile interface devices Braille OS module

1 is a view schematically showing an entire system including a tactile interface device and a computing device according to an embodiment of the present invention.

The tactile interface device 1000 illustrated in FIG. 1 is only one example, and the present invention is not limited thereto. The tactile interface device 1000 connected to the computing device A controlling the tactile interface device of the present invention described below is any interface device capable of providing tactile graphics, and a tactile interface device capable of interacting with a user. , And a computing device capable of outputting tactile graphics.

As shown in FIG. 1, the computing device A is connected to the tactile interface device 1000, and such a connection is a connection including both a wired connection and a wireless connection.

The computing device A includes a smart phone, a tablet personal computer (PC, hereinafter referred to as a 'PC'), a mobile phone, a video phone, and an e-book reader ( e-book readers, desktop PCs, laptop PCs, netbook PCs, personal digital assistants (PDAs, hereinafter referred to as PDAs), A portable multimedia player (PMP, hereinafter referred to as 'PMP'), an MP3 player, a mobile medical device, a camera, and a wearable device (eg, head- Head-mounted devices (HMD, for example, referred to as 'HMD'), electronic clothing, electronic bracelets, electronic necklaces, electronic accessories, electronic tattoos, or smart watches watch).

The computing device A may include one or more processors and memory, and may be selectively connected to a display device such as a monitor or may have its own display module.

Alternatively, the computing device A may be in a form combined with the tactile interface device 1000. In this case, the user may recognize the tactile interface device 1000 and the computing device A controlling the device as one device. In the case of the combined device, the computing device A and the tactile interface device 1000 may share a processor and memory.

The tactile interface device 1000 illustrated in FIG. 1 includes an external device connection unit 1100 capable of performing wired or wireless communication with the computing device A; A tactile display unit 1200 providing tactile information to a user with a plurality of pins; A direction key unit 1300 for changing an input coordinate or a position of a detail focused item; A keyboard unit 1400 capable of receiving information in the form of a key input from a user; And a control unit 1500 that controls operations of the external device connection unit 1100, the tactile display unit 1200, the direction key unit 1300, and the keyboard unit 1400.

The external device connection unit 1100 includes at least one of a communication module unit capable of performing wireless communication and a wired connection unit capable of wiredly connecting to the external device. The communication module unit Bluetooth (bluetooth) communication module, Zigbee (zigbee) communication module, infrared communication module, BLE (bluetooth low energy) communication module, audio communication module, long term evolution (LTE) communication module, and WiFi communication module, IrDA Based infrared communication module, a wireless LAN (WLAN), a WiBro (WiBro) module, a wireless USB (Wireless USB) module, and at least one. Meanwhile, the wired connection unit 1120 preferably includes a connection module using a universal serial bus (USB) interface, and this is a wired connection module capable of transmitting and receiving data.

The tactile display unit 1200 provides tactile information in one or more dimensions of tactile pixels, and the tactile pixels are configured by a plurality of pins that move up and down by applying power to a transducer including a piezoelectric ceramic and an elastic body. . Preferably, the tactile pixel provides tactile information in two dimensions.

The tactile display unit 1200 may include: a display data receiving unit that receives data generated from a tactile interface device based on data transmitted from an external user terminal; A tactile data conversion unit converting the data into tactile display data; And a plurality of pin driving modules driven by the tactile display data. And a driving power supply unit that receives power to drive the tactile display unit 1200, and provides tactile information or tactile graphics based on the received data.

The tactile display unit 1200 may display or provide tactile pixels of one or more dimensions. In one example, the tactile pixel is configured by a plurality of pins moving up and down by applying power to a transducer including a piezoelectric ceramic and an elastic body.

Specifically, the tactile pixels are displayed by a plurality of pin drive module groups, and each of the pin drive module groups consists of a plurality of pin drive modules. Alternatively, the entire pin drive module may constitute one pin drive module group.

On the other hand, the direction key unit 1300 changes the input coordinates or the location of the details to be focused.

The keyboard unit 1400 is a key form that a visually impaired person can input to the tactile interface device 1000, and the keyboard unit 1400 may be composed of a plurality of keys, and input of each key may be performed by the computing. It may be converted into an instruction of an application being executed in the device (A).

On the other hand, the user performs an input to the tactile interface device through the directional key unit 1300 and the keyboard unit 1400, and the tactile interface device 1000 converts the command or information thus input to the user terminal. (A).

Meanwhile, the keyboard unit 1400 preferably includes a braille keyboard that converts braille, which is currently used by the visually impaired, into a general character and transmits a character input signal to the computer.

The keyboard unit 1400 receives the braille and transmits it to the tactile interface device 1000 or the computing device A connected to the tactile interface device 1000. The keyboard unit 1400 may include a braille key, a shortcut key, and an execution or space key.

Since braille is composed of a number of dots to form a single character, if the braille keys are pressed simultaneously, the final braille information can be transmitted. The transmitted braille information may be translated into a general character through software inside the tactile interface device 1000 or software of a computing device.

Therefore, for the computer input / output function for the visually impaired, the tactile display unit 1100 functions as a monitor of a general computer, and the direction key unit 1300 and the keyboard unit 1400 are used as a keyboard, mouse, and the like of the general computer. It acts as an input device.

2 is a diagram schematically showing an internal configuration of a computing device according to an embodiment of the present invention.

The computing device that controls the tactile interface device 1000 according to the present embodiment may include a processor, a bus (corresponding to a double-headed arrow between a processor, a memory, and a network interface unit), a network interface, and a memory. The memory may include a braille OS sub executable code 3400, a general OS sub executable code 3500, an application program sub executable code 3600, and an input / output control routine 3700. The processor includes a braille OS unit 2000; General OS unit 3100; And it may include an application program unit 3200. Here, the general OS unit 3100 corresponds to the main operating system of the computing device, and may include GOOGLE ANDROID OS and MICROSOFT WINDOWS OS.

In other embodiments, the computing device A controlling the tactile interface device 1000 may include more components than those in FIG. 2.

The memory is a computer-readable recording medium, and may include a non-permanent mass storage device such as random access memory (RAM), read only memory (ROM), and a disk drive. In addition, the Braille OS sub-executable code 3400, general OS sub-executable code 3500, application program sub-executable code 3600, and input / output control routine 3700 may be stored in the memory. These software components can be loaded from a computer readable recording medium separate from memory using a drive mechanism (not shown). Such a separate computer-readable recording medium may include a computer-readable recording medium (not shown) such as a floppy drive, disk, tape, DVD / CD-ROM drive, and memory card. In other embodiments, software components may be loaded into memory through network interface 3300 rather than a computer-readable recording medium.

The bus may enable communication and data transfer between components of the computing device controlling the tactile interface device. The bus may be constructed using a high-speed serial bus, parallel bus, storage area network (SAN), and / or other suitable communication technology.

The network interface unit 3300 may be a computer hardware component for connecting the computing device A controlling the tactile interface device 1000 to a computer network. The network interface 3300 may connect the computing device controlling the tactile interface device to the computer network through a wireless or wired connection. A computing device that controls the tactile interface device through the network interface unit 3300 may be wirelessly or wiredly connected to the tactile interface device.

The processor may be configured to process instructions of a computer program by performing input / output operations of a computing device that controls basic arithmetic, logic, and tactile interface devices. Instructions may be provided to the processor by the memory or network interface 3300 and via a bus. The processor includes a braille OS unit 2000; General OS unit 3100; And program code for the application program unit 3200. Such program code can be stored in a recording device such as a memory.

The braille OS unit 2000 may be configured to perform a method of controlling the tactile interface device 1000, which will be described below. In the above-described processor, some components may be omitted, additional components (not shown) may be further included, or two or more components may be combined depending on a method of controlling the tactile interface device.

3 is a diagram schematically showing an operating environment of a braille OS unit according to an embodiment of the present invention.

The general OS unit 3100 corresponds to a software module corresponding to the main operating system of the computing device A, and under the control of the general OS unit 3100, the braille OS unit 2000 and the application program unit 3200 are driven. do. That is, the method for controlling the tactile interface device according to the present invention is implemented by the braille OS unit corresponding to the virtual sub-operating system for the tactile interface device, which operates under the control of the main operating system of the computing device (A). You can.

 The braille OS unit 2000 controls the overall matters related to driving and input / output of the tactile interface device 1000, and at the same time, the application program unit 3200 operating under the control of the general OS unit 3100 and the The input / output between the tactile interface devices 1000 is controlled.

In addition, the braille OS unit 2000 may include an application program for controlling the environment of the tactile interface device 1000 and a built-in basic program capable of inputting and outputting from the tactile interface device 1000.

On the other hand, in the environment of the Braille OS unit 2000, developers develop software or application programs for the visually impaired based on the display of a real computing device, for example, a smart phone display, and the tactile sense of such an application program. Input / output to the interface device 1000 may be performed by the braille OS unit 2000.

That is, the braille OS unit 2000 performing the method of controlling the tactile interface device of the present invention develops software having a general GUI by a general developer, and the developed software is visually impaired through the braille OS unit 2000. It can be implemented in the tactile interface device 1000 in a form that can input and output, it can exhibit the effect of allowing a general software developer to supply a variety of software to the visually impaired.

4 is a diagram schematically showing an internal configuration of a braille OS unit according to an embodiment of the present invention. As shown in FIG. 4, the braille OS unit 2000 includes a trigger module 2100, a main screen module 2200, an environment setting module 2300, a TUI (Tactile User Interface) module 2400, and an internal application module. (2500), TTS (Text To Speech) module 2600, and includes a comment module 2700.

The trigger module 2100 checks the connection between the computing device A and the tactile interface device 1000; And calling the execution of the step of executing the main screen module 2200 when it is determined that the computing device A and the tactile interface device 1000 are interconnected.

The braille OS unit 2000 is basically executed on a computing device that interfaces with a visual display. Therefore, the braille OS unit must be executed in order for the visually impaired to use the tactile interface device by executing each module of the braille OS unit in the computing device.

In general, the execution of an application on a smartphone is executed by a user confirming an application icon or the like in the GUI of the smartphone and performing touch input, but in the case of the visually impaired, such an operation may be difficult.

Since the execution of the braille OS unit must be executed by a visually impaired visually impaired person, unlike the execution of an application executed in a general computing device, the trigger module 2100 operates in the background of the main operating system. When the visually impaired person makes a connection between the tactile interface device and the computing device A (preferably, such a connection is made when the visually impaired person enters a physical button on the tactile interface device), the trigger module 2100 ) Primarily calls execution of the main screen module 2200.

The main screen module 2200, for example, when starting the tactile interface device shown in FIG. 9, displays the main screen of the braille OS unit as in the Windows desktop screen, and selects a module or function desired by the user from the main screen. It can provide a page to perform.

Preferably, the main screen module 2200 simply provides an interface displayed on the computing device, as shown in FIG. 9A. That is, when the main screen module is executed, the visual display device connected to the computing device displays a GUI interface as shown in FIG. 9A. At the same time, the output signal of the GUI interface is converted to the output signal of the tactile interface by the TUI module 2400, and transmitted to the tactile interface device 1000, so that the visually impaired can perceive the main screen tactilely. have. Both an application program embedded in the braille OS unit or an external application program capable of operating under the control of the braille OS unit can be manufactured to provide such a GUI.

Meanwhile, the main screen module 2200 is provided by the braille OS unit 2000. That is, when the braille OS unit 2000 is executed, the main screen module provided primarily by the braille OS unit, that is, a sub-operating system executed under the control of the main operating system, is executed. Alternatively, the main screen module may be executed by the trigger module.

The environment setting module 2300 is a module that provides an interface for setting the environment of the braille OS unit 2000 and / or the environment of the tactile interface device 1000. Similarly, when the environment setting module 2300 is executed, an interface for environment setting is provided in a display device connected to the computing device A, and at the same time, the TUI module 2400 uses the interface as the tactile interface device 1000. Converts the input / output to a possible form.

The TUI module 2400 outputs an output signal for realizing a tactile screen of the tactile interface device 1000 corresponding to a screen that can be displayed on the computing device A by the main screen module or another application application module. Create, and generate an input signal for input to the main screen module 2200 from the user input input from the tactile interface device 1000.

That is, the TUI module 2400 is provided by the main screen module 2200, the braille OS unit 2000, and the GUI of an external application application operated under the control of the main operating system or the built-in application or the tactile interface device ( 1000) to convert to a TUI that can interface. Therefore, if a general developer develops only general software that operates on the computing device A, the interface of the software is converted into a form that can be interfaced by the tactile interface device by the TUI module 2400, and consequently, the visual People with disabilities may use the software.

That is, in the Braille OS unit 2000, the application module execution step is called in response to a user input from the main screen module 2200; And a second TUI step for executing the TUI module provided by the sub-operating system, wherein the second TUI step is the tactile interface device corresponding to a screen that can be displayed on the computing device by the application module. It generates an output signal for realizing the tactile screen of and performs an operation of generating an input signal for input to the application module from a user input input from the tactile interface device.

The built-in application module 2500 may correspond to an application such as an alarm, a clock, a basic document creator, a basic document viewer, etc. Similarly, the built-in application is produced based on a GUI for the general public, but the TUI module ( 2400) can also be used by the visually impaired through the tactile interface device.

The TTS module 2600 performs the TTS function itself on elements displayed or tactile displayed by the main screen module 2200, the environment setting module 2300, or other application modules, or the TTS operating in the main operating system It performs the function of requesting TTS execution while providing text information that is the target of the TTS for the functioning module.

The event module 2700 performs a function of generating an event notification output signal to the tactile interface device based on the connection state between the user terminal and the tactile interface device. For the visually impaired, the tactile interface device that they are interfacing does not easily recognize when the connection with the computing device is released, and in order to solve such a problem, the braille OS unit provides an event module, thereby providing information about the access disorder. It performs the function of immediately transmitting to the tactile interface device.

5 is a view schematically showing the internal configuration of a TUI module according to an embodiment of the present invention. (The drawing is not attached to the word document)

The TUI module generates an output signal for implementing a tactile screen of the tactile interface device 1000 corresponding to a screen that can be displayed on the computing device A by the main screen module 2200 or other application. An output signal generator 2410; An input signal generator 2420 for converting inputs such as key inputs from the tactile interface device into a form that can be input to a main screen module or other application executed in the computing device to generate an input signal; A scaling information load unit 2430 for loading scaling information suitable for pixels of a tactile display unit of a tactile interface device connected in operation of the output signal generating unit; And a buffer output signal generator 2440 that generates a buffer output signal based on the output signal and previous tactile display information of the tactile interface device.

Here, the output signal and the buffer output signal include a control signal for a two-dimensional tactile cell array.

The program module including the main screen module 2200 and the like may convert input / output signals to the tactile interface device through the TUI module 2400.

Meanwhile, the scaling information loading unit 2430 preferably loads the tactile display pixel information by automatically receiving the tactile display pixel information of the corresponding tactile interface device from the connected tactile interface device. Alternatively, such tactile display pixel information may be stored in the memory of the computing device A.

6 schematically illustrates a tactile display screen in a tactile interface device according to an embodiment of the present invention.

As described above, the TUI module 2400 generates an output signal generating unit for generating an output signal for realizing a tactile screen of the tactile interface device 1000 corresponding to a screen that can be displayed on the computing device A. (2410).

Meanwhile, the output signal includes a control signal for a two-dimensional tactile cell array, and the tactile screen of the tactile interface device implemented by the output signal may be displayed on the computing device by the main screen module 2200. A content area (L1) implemented based on the content part of the screen; A cursor area (L2) that implements the position and shape of the current user's cursor; And a page area L3 implemented based on page information of a screen that can be displayed on the computing device by the main screen module.

On the other hand, as shown in Figure 6, the tactile interface device is provided with a variety of physical keys, each physical key is assigned a function to perform a more intuitive function. By performing the function of the TUI module 2400 by the input of the physical key, an instruction of an application operating on the computing device is generated by the TUI module, and the corresponding instruction is input to an application operating on the computing device. Can be.

7 schematically illustrates a process of a method for controlling a tactile interface device according to an embodiment of the present invention.

In step S10, a trigger step is performed in which a visually impaired user calls the execution of the main screen module by turning on the tactile interface device or keying the connection of the tactile interface device to the computing device.

As described above, the trigger module 2100 of the braille OS unit 2000 operates in the background under the control of the main operating system of the computing device. When the above conditions are satisfied, the main screen module ( 2200).

In the case of the visually impaired, since it is more advantageous for the page to start first to be constant, preferably, the trigger step S10 calls execution of the main screen module 2200.

In step S20, execution of the main screen module 2200 is performed. In the GUI display connected to the computing device A by executing the main screen module, the screen of the main screen module 2200 is visually displayed. As described above, since the main screen module 2200 is basically visually displayed, developers can develop the main screen module 2200 or other application applications in a general manner without considering braille or visually impaired persons. . In addition, when a visually impaired user has a problem in using a specific application, a problem that occurs when the visually impaired user uses the application is solved by operating a general interface element (for example, a touch display) connected to the computing device. You can do it.

In step S30, the TUI module 2400 is executed on the main screen displayed by the main screen module 2200 to display the output elements of the main screen module 2200 in the tactile display of the tactile interface device 1000. It is converted to a driving signal of the unit 1200, or an input from the tactile interface device 1000 is converted into a command that can be input to the main screen module 2200. This is referred to as a first TUI step (S30) for convenience.

In step S40, a separate application is executed by the user's manipulation. The separate application may be an application provided by the braille OS 2000 unit or an application provided separately from the braille OS unit 2000.

In step S50, the TUI module 2400 is executed on a screen displayed by the separate application to convert an output element of the screen into a driving signal of the tactile display unit 1200 of the tactile interface device 1000. Or converts input from the tactile interface device into a command that can be input to the application. This is referred to as a second TUI step (S50) for convenience. Here, the second TUI step may also be performed by the TUI module 2400 provided by the braille OS unit 2000.

In step S60, execution of the TTS function for a specific area is called according to the operation of the user's tactile interface device. Preferably, the braille OS unit 2000 determines a portion to be a TTS, and requests that the TTS application operate by the main operating system perform a TTS function.

In step S70, based on the connection state between the computing device and the tactile interface device, an event module that notifies the tactile interface device 1000 is executed.

8 schematically shows a detailed process of a trigger step according to an embodiment of the present invention.

The trigger step may be performed before the step of executing the main screen module 2200. That is, the general user does not call the main screen module 2200 by executing the braille OS unit 2000 on the computing device, but the visually impaired user turns on the power of the tactile interface device (in this case, automatically allocates it). Connected to the computing device) or by operating the tactile display device that is turned on to request access to the computing device.

The triggering step includes: confirming a connection between the computing device A and the tactile interface device 1000 (S11); And invoking the execution of the step of executing the main screen module 2200 when it is determined that the computing device A and the tactile interface device 1000 are interconnected (S12).

9 schematically illustrates a display screen in a computing device and a tactile display in the tactile interface device according to an embodiment of the present invention.

9 (A) shows a display screen of the computing device when the main screen module is executed when the computing device is a smartphone, and FIG. 9 (B) shows a time point when the (A) of FIG. 9 is displayed. The state of the tactile interface device at the same time is shown.

The main screen module 2200 displays each menu (content) in a one-dimensionally arranged form, and displays a cursor for the position next to each menu. That is, the main screen module divides the content into a plurality of units and displays the content on a computing device in a form divided into a plurality of units. Thereafter, the TUI module 2400 divides the content divided into a plurality of units into a plurality of sub-content areas.

On the other hand, the tactile interface device 1000 similarly displays each menu (content) in a one-dimensionally arranged form, and displays the cursor for this to be positioned next to each menu.

10 schematically illustrates a region configuration of a tactile display in a tactile interface device according to an embodiment of the present invention.

The tactile screen of the tactile interface device implemented by the output signal generated by the TUI module 2400 is a content portion of the screen that can be displayed on the computing device A by the main screen module 2200 (FIG. 9, a content area L1 implemented based on a menu); A cursor area L2 that implements the current user's cursor position and shape; And a page area L3 implemented based on page information of a screen that can be displayed on the computing device by the main screen module 2200.

In addition, the content area may be divided into a plurality of sub-content areas, the cursor area may be divided into a plurality of sub-cursor areas, and the position of the cursor implemented in the cursor area is the position of the focused sub-content area. It is desirable to correspond to. This type of tactile display corresponds to a structure in which information for a visually impaired person can be identified and inputted most efficiently.

The sub-content area is composed of a plurality of braille cell groups, and each braille cell group includes a plurality of braille cells, for example, six braille cells. Here, one braille cell refers to one braille hole.

Similarly, each of the sub-cursor regions also includes one or more braille cells, preferably a plurality of braille cells in a form that can inform the location and shape information of the cursor.

More preferably, as in “Focused L2 region” and “Focused L1 region” in FIG. 10, the sub-cursor region and the sub-content regions corresponding to the sub-cursor region are aligned with each other in one axis. In this structure, the visually impaired can most effectively grasp where the cursor is focusing. Such an interface form corresponds to a form derived by various applicants of the present invention through various experiments with the blind.

11 schematically illustrates an input / output process of a tactile interface device according to an embodiment of the present invention.

Steps S110, S120, and S130 are performed by the TUI module 2400. Meanwhile, in step S140, a command is input to the application, and accordingly, the interface or screen output of the application is changed.

Steps S150, S160, and S170 are performed by the TUI module 2400. By performing these steps, a change in the interface or screen output of the application is transmitted to the tactile interface device 1000, and the user can immediately recognize the change.

12 schematically illustrates a process of an output signal generation step according to an embodiment of the present invention.

The output signal generation step includes: a content area output signal generation step of generating a control signal of the two-dimensional tactile cell array for the content area (S310); Cursor area output signal generation step (S320) of generating a control signal of the two-dimensional tactile cell array for the cursor area; and page area output signal generation of generating a control signal of the two-dimensional tactile cell array for the page area Step (S330); includes.

As shown in FIG. 10, the content area may be divided into a plurality of sub-content areas, the cursor area may be divided into a plurality of sub-cursor areas, and the content area output signal generation step (S310) is the main screen module. By separating the data of the content portion of the screen that can be displayed on the computing device, and generates an output signal to implement the plurality of sub-content area, the cursor area output signal generation step (S320) is the plurality of sub In the content area, an output signal is generated to control the tactile cell array at a part of the cursor area corresponding to the sub-content area where the user's cursor is located.

13 schematically shows a process of a cursor region output signal generation step according to an embodiment of the present invention.

The cursor region output signal generation step (S320) includes: determining a location of a user's cursor (S321); And determining the shape of the user's cursor (S322 and S323).

The user's cursor performs a function for focusing a part of the detailed content area. In the case of a general cursor, only the location information needs to be displayed, but in the case of a tactile display device, since a braille pixel has a lower pixel than a normal display pixel, there may be a problem in that information in one detail content area cannot be displayed at once. .

In one embodiment of the present invention, in addition to determining the location of the cursor, by performing the steps (S322 and S323) of determining the shape of the user's cursor, the visually impaired user can use the software intuitively.

Specifically, the step of determining the shape of the user's cursor (S322 and S323) includes a tactile interface of a text in which the cursor is located, or the original text of the sub-content area indicated by the cursor, or the sub-content area indicated by the cursor. Loading the tactile display information from the output signal of the device or the tactile interface device (S322) and determining the cursor shape based on the original text and the output signal or the tactile display information to generate the cursor layer output signal (S323). Includes.

That is, in the cursor area output signal generation step, tactile display information in the tactile interface device of the sub-content area where the user's cursor is located and original information of the main screen module of the sub-content area where the user's cursor is located Based on this, the shape of the user's cursor is determined.

Preferably, in the determining of the shape of the user's cursor (S322 and S323), the tactile display information in the tactile interface device in the sub-content area in which the user's cursor is located is a sub in which the user's cursor is located. If all the original text information of the main screen module in the content area is included, it is determined as the first cursor form,

A second cursor type when the tactile display information in the tactile interface device of the sub-content area where the user's cursor is located includes part of the original text information of the main screen module of the sub-content area where the user's cursor is located To decide.

More preferably, the original text information of the main screen module 2200 of the sub-content area where the user's cursor is located is the first original information, the second original information following the first original information, and the second original information In the case of including the third original text information that follows, the second cursor form may include when the tactile display information in the tactile interface device in the sub-content area where the user's cursor is located corresponds to the first original text information. The displayed 2-1 cursor type; A 2-2 cursor form displayed when tactile display information in the tactile interface device in the sub-content area where the user's cursor is located corresponds to a part of the second original text information; And a 2-2 cursor form displayed when tactile display information in the tactile interface device of the sub-content area where the user's cursor is located corresponds to the third original text information.

In a preferred embodiment of the present invention, information in the content area is divided into a plurality of content sub-areas, and displayed on a tactile interface device.

For example, assume that information in the content area is as follows.

 "The base of a refreshable braille display often integrates a pure braille keyboard.There, the input is performed by two sets of four keys on each side, while output is via a refreshable braille display consisting of a row of electro-mechanical character cells, Other variants exist that use a conventional QWERTY keyboard for input and braille pins for output, as well as input-only and output-only devices. each of which can raise or lower a combination of eight round-tipped pins.

On some models the position of the cursor is represented by vibrating the dots, and some models have a switch associated with each cell to move the cursor to that cell directly.

The mechanism which raises the dots uses the piezo effect of some crystals, whereby they expand when a voltage is applied to them. Such a crystal is connected to a lever, which in turn raises the dot. There has to be a crystal for each dot of the display, i.e. eight per character. "

In this case, the TUI module displays "The base of a refreshable braille display often integrates a pure braille keyboard. There, the input is performed by two sets of four keys on each side, while output is via a refreshable braille display consisting of a row of electro-mechanical character cells, each of which can raise or lower a combination of eight round-tipped pins.Other variants exist that use a conventional QWERTY keyboard for input and braille pins for output, as well as input-only and output-only devices. "as the first sub-content area, to generate an output signal for this,

Output signal for this by assigning "On some models the position of the cursor is represented by vibrating the dots, and some models have a switch associated with each cell to move the cursor to that cell directly." Create

 "The mechanism which raises the dots uses the piezo effect of some crystals, whereby they expand when a voltage is applied to them. Such a crystal is connected to a lever, which in turn raises the dot.There has to be a crystal for each By assigning dot of the display, ie eight per character. "as the third sub-content area, an output signal for this can be generated.

However, when it is said that one sub-content area can display 30 CHARACTER, the original text of the second sub-content area can be classified as follows.

"On some models the position of" (first original information)

"the cursor is represented by v" (the first part of the original text information)

"ibrating the dots, and some mo" (second part of the second original text)

"dels have a switch associated" (second part of the original text information)

"with each cell to move the cur" (4th part of the 2nd original text)

"sor to that cell directly." (3rd original information)

Here, one sub-content area may display only one of the first original text information and the third original text information. In this case, the visually impaired user checks the braille information of the cursor displayed on the sub-cursor area corresponding to the sub-content area (second sub-content area), and the information of the sub-content area currently output from the tactile interface device is among the entire information. You can see where it is located. Thereafter, the visually impaired person can move between the first text information and the third text information by operating the direction keys of the tactile interface device. In this way, the visually impaired user can grasp information more intuitively while minimizing discomfort caused by the limit of the tactile pixel of the tactile display device.

14 schematically shows an example of a form of a cursor area notation according to an embodiment of the present invention.

14 shows an example of a cursor type that can be displayed on a total of four cursor areas, more preferably, a sub-cursor area. By grasping such a cursor shape and operating the arrow keys, the display contents of the sub content layer can be changed.

The first cursor shape, the 2-1 cursor shape, the 2-2 cursor shape, and the 2-3 cursor shape are the first cursor shape, the second cursor shape, and the third cursor shape in FIG. 14, respectively. , Corresponds to the fourth cursor shape.

15 schematically shows some examples of cursor region notation according to an embodiment of the present invention.

As shown in FIG. 15 (A), the entire content is divided into contents 1 to 6, shown in each sub content area, and when the user focuses on the first sub content area, the first sub content area A tactile graphic for the cursor is displayed in the sub-cursor area located side by side or aligned with.

Here, when all of the contents 1 cannot be displayed in one sub-content area, the contents 1 are virtually divided into "content 1A contents 1B contents 1C", and initially, the contents 1A are tactile as shown in FIG. 15B. Graphically, the tactile graphic for the cursor on the left has the second cursor shape in FIG. 14 above.

Then, when the user presses the right arrow key, as shown in FIG. 15 (C), the content 1B is displayed as a tactile graphic, and the tactile graphic for the left cursor has the third cursor form of FIG. 14.

Thereafter, when the user presses the right arrow key, as shown in FIG. 15 (D), the content 1C is displayed as a tactile graphic, and the tactile graphic for the left cursor has the fourth cursor form of FIG. 14.

16 schematically illustrates a process of a buffer output signal generation step according to an embodiment of the present invention.

Hereinafter, for convenience of description, the operation of the buffer output signal generation step for conversion of the main screen module will be described. However, the buffer output signal generation step may be applied to conversion of the display screen of another application as well as conversion of the main screen module.

As described above, the first TUI step (S30) is an output for implementing a tactile screen of the tactile interface device corresponding to a screen that can be displayed on the computing device by the main screen module 2200 or another application. An output signal generation step of generating a signal (S150); And a buffer output signal generation step (S160) of generating a buffer output signal based on the output signal and previous tactile display information of the tactile interface device.

Here, the output signal and the buffer output signal include a control signal for a two-dimensional tactile cell array.

Preferably, the first TUI step (S30) is performed whenever the screen that can be displayed on the computing device is changed by the main screen module, and the buffer output signal generation step is performed immediately before the output. The buffer output signal is generated based on the difference between the signal and the current changed output signal.

Specifically, the buffer output signal generation step (S160) comprises: loading a previous output signal generated by the output signal generation unit before the screen changes to the previous user's input (S410);

Calculating a difference between the current output signal generated by the output signal generation unit and the previous output signal after the screen is changed by the current user input (S420);

And generating a buffer output signal based on the difference between the current output signal and the previous output signal (S430).

Since the buffer output signal includes only the control signal for the currently changed portion of the tactile array in the tactile interface device, the tactile interface device can operate more physically and smoothly.

17 schematically shows an example of a TTS step according to an embodiment of the present invention.

As described above, the method for controlling the tactile interface device 1000 further includes a TTS step of performing a TTS command according to a user's command.

The TTS step includes: determining whether to perform TTS based on an input signal or command from the tactile interface device (S510); Loading information about the currently set TTS mode (S520); A cursor position loading step of loading the current user's cursor position (S530); And determining original text information as a TTS object based on the TTS mode and the location of the user's cursor (S540). And requesting execution of the original text information determined in step S540 to the TTS execution module (S550).

Preferably, the TTS step may be performed for each sub-content area. That is, the original text of the sub-content area in which the current cursor is located may be output in voice by the TTS step.

More preferably, the TTS mode includes a mode for performing TTS on the focused sub-content area; And a mode for performing TTS including the focused sub-content area and a sub-content area following the focused sub-content area.

That is, "On some models the position of the cursor is represented by vibrating the dots, and some models have a switch associated with each cell to move the cursor to that cell directly." Corresponds to the first subcontent area, and "The mechanism which raises the dots uses the piezo effect of some crystals, whereby they expand when a voltage is applied to them.Such a crystal is connected to a lever, which in turn raises the dot.There has to be a crystal for each dot of When the display, ie eight per character. "corresponds to the second sub-content area, and the area where the cursor is focused is the first sub-content area, the TTS mode is a first mode and a first mode that processes only the first sub-content area. It may include a second mode of processing a sub-content area and a second sub-content area, but stopping when a command related to a user's OFF is input.

18 schematically shows an example of an event step according to an embodiment of the present invention.

As shown in FIG. 18, the event step includes: determining a connection state between the user terminal and the tactile interface device (S610); And generating an event output signal to be output to the tactile interface device when it is determined that the connection state corresponds to a preset criterion (S620).

By performing such an event step, the visually impaired user can intuitively determine whether or not the impairment occurs when a connection between the tactile interface device and the computing device occurs.

19 shows a screen for controlling a tactile interface device according to an embodiment of the present invention, FIG. 20 shows a screen for controlling a tactile interface device according to an embodiment of the present invention, and FIG. A screen for controlling a tactile interface device according to an embodiment is illustrated.

19 to 21, a method of controlling a tactile interface device according to an embodiment of the present invention allows any program to intuitively interface with a program for a visually impaired user, and also a program developer For those who do not know braille, etc., and do not develop a separate input / output API, they can supply software for the visually impaired through general software development.

2. Image through tactile interface device viewer  How to provide features

22 schematically shows the configuration of an image viewer module and a tactile interface device according to an embodiment of the present invention.

The configuration of the computing device A shown in FIG. 22 follows the description with reference to FIGS. 2 and 3 above. Meanwhile, the configuration of the braille OS unit 2000 of FIG. 22 follows the description with reference to FIGS. 4 to 18.

Meanwhile, the image viewer module 5000 of FIG. 22 may correspond to one form of the application program unit 3200 of FIG. 3, or may correspond to the built-in application module 2500 of FIG. 4.

Similarly, the information output from the image viewer module 5000 may be displayed in a visual form that can be recognized by the general public on a display device connected to the computing device A, and similarly embedded in the braille OS unit 2000 The input / output signal may be converted into a form in which a visually impaired user can interface in the tactile interface device 1000 by a special TUI module for a common TUI module or an image viewer module embedded in the image viewer module 5000.

Preferably, in one embodiment of the present invention, the braille OS unit 2000 includes a common TUI module 2400, and the image viewer module 5000 includes a visually impaired person considering the specificity of the image viewer module. It is preferable to include a separate TUI module (not shown) that can make the interface more smoothly.

Similarly, the execution of the image viewer module 5000 is controlled by the braille OS unit 2000, while the image viewer module 5000 is controlled, the TTS module 2600, the event module 2700, etc. This can also be done concurrently.

Meanwhile, in the computing device A, the image viewer module 5000 is executed, and any one of a main screen step, an image selection step, and an image output step may be performed. Accordingly, detailed processes of any one of the main screen step, the image selection step, and the image output step may be performed according to the input from the computing device A or the input from the tactile interface device 1000.

When the image viewer module 5000 is executed, a screen according to the corresponding step is displayed on the computing device A, and the screen displayed on the computing device A is converted into an output signal to the tactile interface device 1000. Thus, a tactile display corresponding to a screen displayed on the computing device A may be displayed on the tactile interface device 1000.

The image viewer module 5000 may be located inside or connected to the image database 7000 provided as a separate device. One or more image files that can be output through the computing device A or the tactile interface device 1000 may be stored in the image database 7000. In one embodiment of the present invention, the image viewer module 5000 loads an image file stored from the image database 7000 according to a user's selection and outputs it through the computing device A or the tactile interface device 1000. can do.

Although only one image database 7000 is shown in FIG. 22, in one embodiment of the present invention, the image viewer module 5000 may be connected to two or more image databases 7000, and at this time, the image viewer The module 5000 may store a list of two or more image databases 7000.

23 schematically shows the internal configuration of an image viewer module according to an embodiment of the present invention.

As described above, the image viewer module 5000 is implemented as a computing device A including a processor, and is connected to the computing device A through a tactile interface device 1000 capable of interacting with a user. It provides an image viewer function.

On the other hand, the image viewer module 5000 is a TUI module 5100 that converts signals input / output from the internal function module of the image viewer module into a form that can be more intuitively recognized or input by the tactile interface device 1000. (The TUI module is preferably a module included in the image viewer module separately from the TUI module 2400 of the braille OS unit 2000 of FIG. 4); An image selection unit 5200 for selecting an image to be output by the user's input; And an image output unit 5300 outputting the selected image through the computing device and the tactile interface device. It includes.

The TUI module 5100 converts information that can be graphically displayed by the image selection unit 5200 and the image output unit 5300 into a form that can be displayed by a tactile interface in the tactile interface device 1000.

Meanwhile, the TUI module 5100 may include a first output signal generator 5111 capable of converting a display screen into an output signal to a tactile interface device in a first method; And a second output signal generator 5112 capable of converting the display screen into an output signal to the tactile interface device in a second method. This is different from the general TUI module 2400 provided by the braille OS unit 2000 of FIGS. 4 and 5, and it is not converted into the same format for all information, but according to the category of information due to the specificity of the image. By controlling the tactile cell in different ways, it is possible to provide the image viewer function more intuitively to the visually impaired.

First, when the image viewer module 5000 is driven, first of all, the image selector 5200 is executed in the computing device A. The image selection unit 5200 corresponds to a component that allows a user to select an image stored in the image database 7000 to directly provide the selected image to the visually impaired through the tactile interface device 1000. . When the user selects an image through the image selection unit 5200, an image output unit 5300 is executed to output the selected image to the computing device A and the tactile interface device 1000.

When the image output unit 5300 is executed in the computing device A, it outputs the selected image to the visual display device connected to the computing device A, and the TUI module 5100 in the computing device A. ) Is executed, and outputs an image by generating an output signal for realizing a tactile screen of the tactile interface device 1000 corresponding to a screen displayed on the display device by the image output unit 5300.

24 schematically shows detailed steps of the operation of the image selection unit according to an embodiment of the present invention.

Referring to Figure 24, the image selector 5200 according to an embodiment of the present invention includes an image database list display step (S710); Image database selection step (S720); Image category display step (S730); Image category selection step (S740); Image search step (S750); Image list display step (S760); And a display image selection step (S770); An image may be selected by performing an image selection step including.

In one embodiment of the present invention, when the image selector 5200 is executed, an image database list display step S710 of displaying a list of one or more image databases 7000 in which images are stored is performed. At this time, the TUI module 5100 may serve to convert the input / output signal of the image selection unit 5200 into the input / output form of the tactile interface device 1000. The role of the TUI module 5100 is performed in the same manner in all the following steps to control input / output in the tactile interface device 1000.

In one embodiment of the present invention, the list of the image database 7000 may be stored in the image selection unit 5200, and the list of the image database may include the image viewer module 5000 and the computing device (A). It contains information about all internal and external accessible image databases (7000). In one embodiment of the present invention, the image selection unit 5200 may continuously add or delete information about an accessible image database by updating the list of the image database according to a predetermined time interval. Alternatively, the image selection unit 5200 may update the list of the image database whenever the image database list display step S710 is performed.

Thereafter, an image database selection step (S720) of selecting an image database from which an image is to be loaded is performed by a user input. In the image database selection step (S720), a user can select a desired image database from the list of the image database 7000 displayed in the image database list display step (S710) through the tactile interface device 1000. The user may move the cursor located in the list of the image database using the directional keys of the tactile interface device 1000, and select the image database where the cursor is located using the enter key or the like. Such an input and selection process may be performed by the TUI module 5100.

Thereafter, an image category display step S730 of displaying a category of the image stored in the image database is performed. In one embodiment of the present invention, stored images may be classified into categories in the image database. The category may be classified according to the subject of the image, or may be classified according to the file format of the image. For example, the category may be science, Korean language, mathematics, English, Braille, etc. By storing the images stored by the above categories, it is possible to exert the effect that the user can easily access the images desired to be displayed.

Thereafter, an image category selection step (S740) of selecting a category by the user's input or an image search step (S750) of searching for an image by the user's input may be selectively performed.

In the image category selection step (S740), a user's desired category is selected from categories of images displayed in the image category display step (S730). This is similar to the image database selection step (S720), the user can select a desired category from the displayed category of the image through the tactile interface device 1000.

In one embodiment of the present invention, when a category is selected as described above, an image list display step S760 of displaying a list of images in the category may be performed. On the other hand, in another embodiment of the present invention, a sub-category may be included in the category, and may be classified to include an image in the sub-category. In the case of such an embodiment, when selecting a category through the image category selection step S740, the image category display step S730 may be performed again to display a list of sub-categories in the selected category.

In the image search step (S750), a user may search for an image stored in the image database 7000 through a search word or the like. In one embodiment of the present invention, the user may directly enter a search term or select a search term from a list of stored search terms to perform a search.

In one embodiment of the present invention, the image search step (S750) may be performed after the image category display step (S740) is performed, or may be performed after the image list display step (S760) is performed. have. When the image search step (S750) is performed after the image category display step (S740) is performed in one embodiment of the present invention, all images corresponding to the search term of the image search step (S750) are searched for, If a category is selected by the image category selection step (S740), and the image search step (S750) is performed after the image list display step (S760) is performed, the image corresponding to the selected category is the Only the image corresponding to the search word in the image search step S750 may be searched.

Thereafter, an image list display step (S760) of displaying a list of outputable images based on at least one of the selected image database, the selected category, and the image search result is performed. In an embodiment of the present invention, when the user selects a category through the image category selection step (S740) and the image list display step (S760) is performed, it corresponds to the category based on the selected category from the selected image database. Display a list of images to be performed, and perform a search through the image search step (S750), and when the image list display step (S760) is performed, a list of images corresponding to the search results among the selected image databases is displayed. When a category is selected through the image category selection step (S740) and a search is performed through the image search step (S750), when the image list display step (S760) is performed, a search result among selected categories of the selected image database is performed. A list of corresponding images can be displayed.

In one embodiment of the present invention, after the image list display step (S760), the image search step (S750) may be performed again by the user's selection.

Thereafter, a display image selection step (S770) of selecting an image to be output by a user input is performed. In the display image selection step (S770), a user's desired image is selected from the image list displayed in the image list display step (S760). This allows the user to select a desired image from the image list displayed similarly to the image database selection step (S720) and the image category selection step (S740) through the tactile interface device 1000.

The image selected as described above may be output through the display device connected to the computing device A and the tactile interface device 1000 by the image output unit 5300.

25 illustrates a screen of an image database list display step and a tactile interface device in which TUI conversion is performed on the image database list display step according to an embodiment of the present invention.

25A is a screen output through a display device connected to the computing device A, and FIG. 25B is a screen of the tactile interface device 1000 corresponding to FIG. 25A. . Referring to (A) of FIG. 25, the image database list display step (S710) is performed by the image selection unit 5200 to display a list of image databases 7000 connected to the image viewer module 5000. Can be confirmed. In the embodiment of FIG. 25, two image databases (Device storage and ImageBank storage) are displayed in the list.

FIG. 25B shows a state in which the list of such image databases is translated into Braille and displayed on the tactile display of the tactile interface device. At this time, the tactile display displayed by the main screen step and the image selection step is a cursor that tactilely displays information about the current cursor position among the positions of the plurality of cursors as shown in FIGS. 6 and 10. Area L12; A content area (L11) tactilely displaying information corresponding to the position of each cursor; And a page area L13 tactilely displaying information on the current page. It may include. As described above, the area of the tactile display is divided and information is displayed, so that the visually impaired can easily recognize the content of the content, the location of the current cursor, information on the page, and the like, and clearly recognize the information.

26 shows a screen of an image category display step and a tactile interface device in which TUI conversion is performed on the image category display step according to an embodiment of the present invention.

26A is a screen output through a display device connected to the computing device A, and FIG. 26B is a screen of the tactile interface device 1000 corresponding to FIG. 26A. . Referring to (A) of FIG. 26, it can be seen that the image category display step (S730) is performed by the image selector 5200 to display a list of categories in the image database 7000. In the embodiment of Fig. 26, seven categories of science, Korean, other, mathematics, English, Sailing, and Braille are included in the category list. At this time, when the "Search" menu is inserted at the top of the list to select it, an image search step (S750) is performed. When a category is selected, an image list display step (S760) for the selected category is performed. do.

26 (B) shows a state in which a list of such categories is translated into Braille and displayed on the tactile display of the tactile interface device. At this time, the tactile display may be divided into a content area L11, a cursor area L12, and a page area L13 as shown in FIG. 25B.

27 illustrates a screen of an image list display step and a tactile interface device in which TUI conversion is performed on the image list display step according to an embodiment of the present invention.

27A is a screen output through a display device connected to the computing device A, and FIG. 27B is a screen of the tactile interface device 1000 corresponding to FIG. 27A. . Referring to (A) of FIG. 27, it is confirmed that the image list display step (S760) is performed by the image selector 5200 to display a list of images in the category. In the embodiment of Fig. 27, four images of book, edit, envelope, and folder are included in the image list. At this time, the "search" menu may be inserted at the top of the list as shown in FIG. 26, and when this is selected, an image search step (S750) is performed, and when an image is selected, the image output unit 5200 ), The selected image is output through the display device connected to the computing device A and the tactile interface device 1000.

FIG. 27 (B) shows a state in which the list of images is translated into Braille and displayed on the tactile display of the tactile interface device. At this time, the tactile display may be divided into a content area L11, a cursor area L12, and a page area L13 as shown in FIG. 25B.

28 schematically shows detailed steps of an image output step according to an embodiment of the present invention.

Referring to FIG. 28, the image output unit 5300 according to an embodiment of the present invention includes an image discrimination step (S810); Image simplification step (S820); Image display step (S830); And an output image adjustment step (S840); An image may be selected by performing an image output step including.

In an embodiment of the present invention, when an image is selected by the image selection unit 5200 and the image output unit 5300 is executed, an image discrimination step of determining an image type based on information of an image to be output (S810) ) Is performed. The stored image may have a different image format, such as a vector image or a raster image, and a file format of an image such as JPG, GIF, or PNG may be different from a raster image. In order to perform processing on various images, the image discrimination step S810 determines the type of the selected image and selects and processes an image processing algorithm suitable for the determined type.

Thereafter, an image simplification step (S820) of simplifying the image to be output by the tactile interface device based on the determined type of the image is performed. Since the tactile interface device 1000 has a lower resolution and a difficulty in expressing colors compared to a general visual display device, in the image simplification step (S820), the image is in a form suitable for outputting from the tactile interface device 1000. It can be transformed. In one embodiment of the present invention, the grayscale image is generated by grayscale the image, and the grayscale image is binarized to generate a binarized image, so that the tactile interface apparatus 1000 can simplify the output. .

Thereafter, an image display step S830 of outputting the simplified image through the computing device and the tactile interface device 1000 is performed. In the image display step (S830), an output area to be output from the image through the tactile interface device 1000 is determined, and the image of the output area is matched to the pin of the tactile interface device 1000, thereby pinning each pin. Can be driven.

Thereafter, an output image adjustment step (S840) is performed in which the image output through the tactile interface device 1000 is enlarged, reduced, or moved by the user's input, and outputted. When the visually impaired person senses the image output through the tactile interface device 1000 by touching it by hand, it is necessary to enlarge the image to check the image in more detail or move it to another area of the image to detect it. have. To this end, in the output image adjustment step (S840), a visually impaired user may adjust the output image by inputting the enlarged, reduced, or moved image.

29 schematically shows detailed steps of an image simplification step and an image display step according to an embodiment of the present invention.

Referring to FIG. 29, the image simplification step (S820) includes: converting the image to grayscale to generate a grayscale image (S821); And an image binarization step of generating a binarized image by binarizing the grayscale image (S822). Including, the image display step (S830), the output area determining step of setting the area to be output through the tactile interface device 1000 of the image as an output area (S831); A first display information generating step (S832) of generating first display information including information of the image to be displayed on the computing device A based on the image; And a second display information generating step of generating second display information including image information of an output region of a simplified image to be tactilely displayed by the tactile interface device 1000 based on the image (S833); It may include.

In one embodiment of the present invention, the grayscale conversion step (S821) converts the image to grayscale based on the type of the image determined in the image determination step (S810).

Thereafter, in the image binarization step (S822), the grayscale image is binarized to generate a binarized image. The tactile display unit 1200 of the tactile interface device 1000 is composed of a plurality of pins moving up and down, and expresses an image by raising or falling the plurality of pins. In order to determine the rise or fall of the pin, the grayscale image is binarized to generate a binarization image, and the rise or fall of the pin is determined based on the information of the binarization image.

In one embodiment of the present invention, an OTSU algorithm can be used to binarize an image. The OTSU algorithm performs binarization by separating the gray value of a grayscale image based on a threshold value, and uses a statistical method to determine the threshold value.

As described above, in the image simplification step (S820), a grayscale image is generated by converting the image to grayscale, and the grayscale image is binarized to generate a binarized image.

In one embodiment of the present invention, the output area determination step S831 sets an area to be output through the tactile interface device 1000 among the selected images as an output area. In the case of the tactile interface device 1000, since a tactile display is formed by driving using a pin, it has a lower resolution than a general visual display device. Therefore, when the image is output through the tactile interface device 1000, the pixels of the image and the pins of the tactile interface device 1000 are converted to one-to-one in accordance with the resolution of the original image, and the output of the image is extremely high. It is difficult to grasp the image by displaying only a part.

Therefore, when the image is output through the tactile interface device 1000, it is necessary to adjust the size of the image and output the image. For this purpose, an output area is set in the image, and the image in the output area is the tactile interface device ( 1000) can be output according to the size of the tactile display. The output area may include all of the image, or may include a part of the image to be output through the tactile interface device 1000.

When an image is selected through the image selection step in an embodiment of the present invention, the output area 50 may be set as the entire area of the image. In this way, the entire image is initially set as the output area 50, so that the user detects the entire image through the tactile interface device 1000, and then displays the image through the output image adjustment step (S840). By zooming in, out, or moving, you can sense the image in more detail to improve the understanding of the overall image.

In addition, in one embodiment of the present invention, the output area 50 may have an aspect ratio corresponding to the aspect ratio of the content area of the tactile interface device 1000. Thus, by setting the aspect ratio of the output area 50 in accordance with the aspect ratio of the content area that is converted to the image of the output area 50 is output to the tactile display, through the tactile interface device 1000 The converted and output image can be prevented from being distorted.

In the first display information generating step (S832), first display information including information of the image to be displayed on the computing device A is generated based on the image. The first display information is output through a visual display device connected to the computing device (A) so that a visually identifiable image can be checked when used by a visually impaired person, and the information to be output through the tactile interface device 1000 is displayed. It becomes the basis.

In the second display information generating step (S833), second display information including image information of an output area of a simplified image to be tactilely displayed in a tactile interface device is generated based on the image. In an embodiment of the present invention, an image is output through a visual display device connected to the computing device A, and a tactile interface is easily generated by generating a signal capable of realizing the tactile screen of the tactile interface device 1000. The device 1000 can be controlled. At this time, since the second display information has a difference between the size and resolution of the image corresponding to the output area 50 and the size and resolution of the tactile screen of the tactile interface device 1000, interpolation is used. 2Display information can be generated.

In one embodiment of the present invention, in the second display information generation step, the information of the output region of the binarized image is mapped to the coordinates of the pin in a plurality of pins of the tactile interface device 1000, respectively, The second display information may be generated by determining the rising or falling.

According to an embodiment of the present invention, an image can be output through a smart braille device based on a multi-array tactile cell by simplifying an image through grayscale and binarization.

30 schematically shows detailed steps of an output image adjustment step according to an embodiment of the present invention.

Referring to FIG. 30, in the output image adjustment step according to an embodiment of the present invention, the output image enlargement / reduction step of enlarging or reducing the image tactilely displayed by the tactile interface device 1000 to output it is reduced ( S841); And an output image moving step (S842) of moving and outputting the position of the image tactilely displayed by the tactile interface device 1000; It may include.

In the output image enlargement / reduction step (S841), an image displayed on the tactile interface device 1000 is enlarged or reduced according to a user's input to be output. In one embodiment of the present invention, the output image enlargement / reduction step S841 may be performed by adjusting the size of the output area 50. When the size of the output area 50 is reduced, the image in the output area 50 becomes larger than that of the output area 50 and is largely displayed on the tactile interface device 1000 to show an enlarged effect. When the size of the output area 50 is enlarged, the image in the output area 50 becomes smaller than the output area 50 and is reduced by being displayed smaller in the tactile interface device 1000 to reduce the effect. Can be seen. In one embodiment of the present invention, after changing the output area 50 as described above, the computing device is performed by performing the same steps as the first display information generation step (S832) and the second display information generation step (S383). (A) and the output of the tactile interface device 1000 can be updated.

In one embodiment of the present invention, the user may input the image to enlarge / reduce the image using the SHIFT key and the direction key of the tactile interface device 1000. For example, if the left (LEFT) or down (DOWN) key among the SHIFT key and the direction keys is input at the same time, the image is reduced, and if the right (RIGHT) or up (UP) key among the SHIFT key and the direction keys is input at the same time, the image Can be enlarged.

In the output image moving step S842, an image displayed on the tactile interface device 1000 is moved and output according to a user's input. In one embodiment of the present invention, the output image moving step S842 may be performed by moving the output area 50. In one embodiment of the present invention, after changing the output area 50 as described above, the computing device is performed by performing the same steps as the first display information generation step (S832) and the second display information generation step (S383). (A) and the output of the tactile interface device 1000 can be updated.

In one embodiment of the present invention, the user may input to enlarge / reduce the image using the directional keys of the tactile interface device 1000. For example, if a LEFT key is input among the direction keys, it is left, if a DOWN key is entered, and if a RIGHT key is input, to the right, and an UP key is input. You can make the image move upward.

31 illustrates a screen of an output image enlargement / reduction step and a tactile interface device in which TUI conversion is performed on the output image enlargement / reduction step according to an embodiment of the present invention.

31A is a screen output through a display device connected to the computing device A, and FIG. 31B is a screen of the tactile interface device 1000 corresponding to FIG. 31A. , FIG. 31 (C) is a screen of the tactile interface device 1000 that is output by reducing the output image of FIG. 31 (B).

31 (A) shows a state in which an image of a hand selected through the image selection unit 5100 is output through a display device connected to the computing device A. The image is converted into information capable of raising or lowering each of the pins of the tactile interface device 1000 as shown in FIG. 31B through the image simplification step (S820) and the tactile interface device 1000. Is displayed on the tactile display of. As described in FIG. 25, the tactile display includes a cursor area L22 tactilely displaying information on a current cursor position among a plurality of cursor positions; A content area (L21) tactilely displaying information corresponding to the position of each cursor; And a page area (L23) tactilely displaying information on the current page. It may include. In FIG. 31B, an image of the hand corresponding to FIG. 31A is displayed in the content area L21.

In one embodiment of the present invention, information of the image may be output to the page area L23. For example, the file name of the image may be displayed in Braille in the page area L23.

At this time, the user may enlarge or reduce the image by operating the tactile interface device 1000. FIG. 31 (C) shows the result of reducing the output image as shown in FIG. 31 (B). At this time, in one embodiment of the present invention, the image may be reduced by enlarging the output region 50 so that more images are included in the output region 50. 31 (C), the image of the reduced hand is output through the tactile display of the tactile interface device 1000 as compared to FIG. 31 (B).

32 illustrates a screen of a tactile interface device in which a TUI conversion is performed on an output image moving step and the output image moving step according to an embodiment of the present invention.

32A is a screen output through a display device connected to the computing device A in the same manner as in FIG. 31A, and FIG. 32B is the screen corresponding to FIG. 32A. It is a screen of the tactile interface device 1000, and FIG. 32 (C) is a screen of the tactile interface device 1000 output by moving the output image of FIG. 26 (B).

32 (A) and 32 (B) are the same as FIG. 31 (A) and FIG. 32 (B). The selected image is converted into information capable of raising or lowering each pin of the tactile interface device 1000 through the image simplification step (S820), as shown in FIG. 32 (B), so that the tactile interface device 1000 It is displayed on the tactile display. The tactile display includes a cursor area L22 tactilely displaying information on a current cursor position among a plurality of cursor positions; A content area (L21) tactilely displaying information corresponding to the position of each cursor; And a page area (L23) tactilely displaying information on the current page. It may include. In FIG. 32B, an image of the hand corresponding to FIG. 32A is displayed in the content area L21.

The user may move the image by operating the tactile interface device 1000. 32 (C) shows the result of moving the output image to the left as shown in FIG. 32 (B). At this time, in one embodiment of the present invention, the output area 50 is moved to the right, so that the right area of the hand displayed on the image is included more, so that the image can be moved to the left. 32 (C), an image in which the wrist portion is covered and only the remaining hand is displayed is output through the tactile display of the tactile interface device 1000 as compared to FIG. 31 (B).

According to an embodiment of the present invention, it is possible to enlarge, reduce, and move an image output through a multi-array tactile cell-based smart braille device, thereby exerting the effect of allowing the visually impaired to interpret the image file in detail.

33 illustrates a screen of a tactile interface device in which TUI conversion is performed on an output image adjustment step and the output image adjustment step according to an embodiment of the present invention.

33A is a screen output through a display device connected to the computing device A, and FIG. 33B is a screen of the tactile interface device 1000 corresponding to FIG. 33A. .

Referring to (A) of FIG. 33, an image of a book is output through a display device connected to the computing device (A), and referring to (B) of FIG. 33, a partial area (output area, 50) of the image of the book ) Is output through the tactile interface device 1000. At this time, the tactile display displayed through the tactile interface device 1000 is enlarged and displayed only in a part corresponding to the output area 50 (part of A of the book cover and the upper left of the book) rather than all of the images of the book. .

At this time, according to an embodiment of the present invention, the tactile display displayed by the image output step S800 is a content area L31 that tactilely displays image information of the output area 50 of the simplified image. ); And a location information area (L34) tactilely displaying information on the output area (50). It may include. In (B) of FIG. 33, a rectangular location information area L34 is located at the upper right, and the position of the output area 50 with respect to the entire image is displayed. As shown in (A) of FIG. 33, the output area 50 is located on the upper left side with respect to the center of the image, and the position and size of the output area 50 are located in the location information area L34. Is displayed, so that the user can confirm the output area 50 displayed through the tactile interface device 1000 at a certain size and location of the image.

In addition, in one embodiment of the present invention, the first display information further includes information of the output area 50, and the image and the output are displayed on the computing device A displayed by the image output step S800. Area 50 may be displayed. This is displayed on the tactile interface device 1000 by displaying the output area 50 together when displaying the image on the display device connected to the computing device A as shown in FIG. 33A. It is possible to check which area of the image corresponds to the area being used. The output area 50 may be displayed as a line as shown in FIG. 33A.

According to an embodiment of the present invention, when the output image is enlarged or moved, it is possible to visually impair the interpretation of the image file by allowing the visually impaired user to check the location of the location in the entire image. Can be effective.

34 and 35 illustrate screens of an image output step and a tactile interface device in which TUI conversion is performed on the image output step according to an embodiment of the present invention.

In one embodiment of the present invention, in the second display information generation step (S833), when mapping the information of the output area 50 of the binarized image to the coordinates of the pin, in the area matching the coordinates of the pin If the proportion of black is 50% or more, the pin is fully raised, and if the proportion of black in the area matching the coordinates of the pin is less than 20%, the pin is completely lowered, and the proportion of black in the area matching the coordinates of the pin is If 20% or more and less than 50%, the second display information 50 may be generated to increase the pin by 40 to 60%.

34A is an image to be output through the tactile interface device 1000. Since the tactile interface device 1000 has a lower resolution of an area that can be output than a general visual display device, in an image composed of lines as shown in FIG. 34A, in the case of lines having different thicknesses, the coordinates of the pins are simply used. When matching, it is difficult to show the difference.

In order to improve this, in one embodiment of the present invention, the height of the pins may be varied according to the ratio of the black of the region matching the coordinates of the pins to display the image more precisely through the tactile interface device 1000.

FIG. 34 (B) shows the result of matching the image shown in FIG. 34 (A) to the pin of the tactile interface device 1000. Referring to (B) of FIG. 34, a pin matching a thick line area is shown in black, and a pin matching a thin line area is shown in gray. At this time, in the case of a pin shown in white, it is completely lowered to enter the tactile display plate 1210, and in the case of a pin shown in black, the pin 1220 is completely raised as shown in FIG. In the case of a pin shown in gray, as shown in FIG. 34 (C), the pin 1220 rises 40 to 60% to be positioned in the middle.

In another embodiment of the present invention, in the second display information generation step (S833), the coordinates of the pin are the information of the output area 50 of the grayscale image in a plurality of pins of the tactile interface device 1000. The second display information may be generated by determining the step of rising or falling each of the pins by mapping to the.

35A is an image to be output through the tactile interface device 1000. The image shown in (A) of FIG. 35 is a color image, and it is necessary to perform an image simplification step (S820) in order to be output through the tactile interface device 1000.

At this time, in one embodiment of the present invention, the image is output through the tactile interface device 1000 based on the grayscale image, not the binarized image generated in the image simplification step S820 to display the image in more detail. You can do it.

FIG. 35 (B) shows a grayscale image obtained by gray-scaling the image shown in FIG. 35A, and in FIG. 35C, the grayscale image is the tactile interface device 1000. The result of matching the pins of is shown. The grayscale image of FIG. 35B is a four-level grayscale image (white, light gray, dark gray, black), and each color step of the grayscale image is matched with FIG. 35C, respectively. Black for pins matching the black area, dark gray for pins matching the dark gray area, light gray for pins matching the light gray area, and white for pins matching the white area. It is done.

At this time, in the case of a pin shown in white, the pin is completely lowered to enter the tactile display plate 1210, and in the case of a pin shown in light gray, the pin 1220 is 30 to 30 as shown in FIG. The pin 1220 rises 60 to 70% and rises 60 to 70% as shown in FIG. In this case, as shown in FIG. 34 (F), the pin 1220 may be completely raised.

In the present invention, by controlling the heights of the pins of the tactile interface device 1000 differently as in the embodiments of FIGS. 34 and 35, an image can be displayed through the tactile interface device 1000 in more detail.

According to an embodiment of the present invention, the effect of allowing the visually impaired to interpret the image file at a level similar to the general public through the tactile sense of the stored image based on the multi-array tactile cell-based smart braille device and the visually impaired TUI. Can be exercised.

36 is a diagram illustrating an example of an internal configuration of a computing device according to an embodiment of the present invention.

As illustrated in FIG. 36, the computing device 11000 includes at least one processor 11100, a memory 11200, a peripheral interface 11300, and an input / output subsystem ( I / O subsystem) 11400, a power circuit 11500, and a communication circuit 11600. At this time, the computing device 11000 may correspond to the user terminal A connected to the tactile interface device or the aforementioned computing device A.

The memory 11200 may include, for example, high-speed random access memory, magnetic disk, SRAM, DRAM, ROM, flash memory, or non-volatile memory. have. The memory 11200 may include software modules, instruction sets, or other various data necessary for the operation of the computing device 11000.

At this time, access to the memory 11200 from other components such as the processor 11100 or the peripheral interface 11300 may be controlled by the processor 11100.

The peripheral interface 11300 may couple input and / or output peripherals of the computing device 11000 to the processor 11100 and the memory 11200. The processor 11100 may execute various functions for the computing device 11000 and process data by executing a software module or a set of instructions stored in the memory 11200.

The input / output subsystem 11400 can couple various input / output peripherals to the peripheral interface 11300. For example, the input / output subsystem 11400 may include a controller for coupling a peripheral device, such as a monitor or keyboard, mouse, printer, or touch screen or sensor, to the peripheral interface 11300, as needed. According to another aspect, I / O peripherals may be coupled to the peripheral interface 11300 without going through the I / O subsystem 11400.

The power circuit 11500 may supply power to all or part of the components of the terminal. For example, the power circuit 11500 may include a power management system, one or more power sources such as a battery or alternating current (AC), a charging system, a power failure detection circuit, a power converter or inverter, a power status indicator, or power. It can include any other components for creation, management, and distribution.

The communication circuit 11600 may enable communication with other computing devices using at least one external port.

Alternatively, as described above, if necessary, the communication circuit 11600 may enable communication with other computing devices by transmitting and receiving an RF signal, also known as an electromagnetic signal, including an RF circuit.

The embodiment of FIG. 36 is only an example of the computing device 11000, and the computing device 11000 may omit some components illustrated in FIG. 36, or further include additional components not illustrated in FIG. 36, or 2 It may have a configuration or arrangement that combines more than one component. For example, a computing device for a communication terminal in a mobile environment may further include a touch screen or a sensor, etc. in addition to the components illustrated in FIG. 36, and various communication methods (WiFi, 3G, LTE) in the communication circuit 1160. , Bluetooth, NFC, Zigbee, etc.) may include a circuit for RF communication. Components that may be included in the computing device 11000 may be implemented in hardware, software, or a combination of both hardware and software including integrated circuits specialized for one or more signal processing or applications.

Methods according to an embodiment of the present invention may be recorded in a computer readable medium by being implemented in the form of program instructions that can be performed through various computing devices. In particular, the program according to the present embodiment may be configured as a PC-based program or an application dedicated to a mobile terminal. The application to which the present invention is applied may be installed on a user terminal through a file provided by a file distribution system. For example, the file distribution system may include a file transmission unit (not shown) that transmits the file according to a request of the user terminal.

The device described above may be implemented with hardware components, software components, and / or combinations of hardware components and software components. For example, the devices and components described in the embodiments include, for example, processors, controllers, arithmetic logic units (ALUs), digital signal processors (micro signal processors), microcomputers, field programmable gate arrays (FPGAs). , A programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions, may be implemented using one or more general purpose computers or special purpose computers. The processing device may run an operating system (OS) and one or more software applications running on the operating system. In addition, the processing device may access, store, manipulate, process, and generate data in response to the execution of the software. For convenience of understanding, a processing device may be described as one being used, but a person having ordinary skill in the art, the processing device may include a plurality of processing elements and / or a plurality of types of processing elements. It can be seen that may include. For example, the processing device may include a plurality of processors or a processor and a controller. In addition, other processing configurations, such as parallel processors, are possible.

The software may include a computer program, code, instruction, or a combination of one or more of these, and configure the processing device to operate as desired, or process independently or collectively You can command the device. Software and / or data may be interpreted by a processing device, or to provide instructions or data to a processing device, of any type of machine, component, physical device, virtual equipment, computer storage medium or device. , Or may be permanently or temporarily embodied in the transmitted signal wave. The software may be distributed on networked computing devices and stored or executed in a distributed manner. Software and data may be stored in one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer readable medium. The computer-readable medium may include program instructions, data files, data structures, or the like alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the embodiments or may be known and usable by those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs, DVDs, and magnetic media such as floptical disks. -Hardware devices specially configured to store and execute program instructions such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include high-level language code that can be executed by a computer using an interpreter, etc., as well as machine language codes produced by a compiler. The hardware device described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

As described above, although the embodiments have been described by a limited embodiment and drawings, those skilled in the art can make various modifications and variations from the above description. For example, the described techniques are performed in a different order than the described method, and / or the components of the described system, structure, device, circuit, etc. are combined or combined in a different form from the described method, or other components Alternatively, even if replaced or substituted by equivalents, appropriate results can be achieved.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims (10)

A method implemented as a computing device including a processor and connected to the computing device to provide an image viewer function through a tactile interface device capable of interacting with a user,
In the computing device, an image viewer module is executed, and one of the main screen step, the image selection step, and the image output step is performed,
When the image viewer module is executed, a screen according to the corresponding step is displayed on the computing device, and the screen displayed on the computing device is converted into an output signal to the tactile interface device, corresponding to the screen displayed on the computing device. A tactile display can be displayed on the tactile interface device,
Depending on the input from the computing device or the input from the tactile interface device, detailed processes of any one of the main screen step, the image selection step and the image output step are performed,
The image output step,
An image discrimination step of determining the type of the image based on the information of the image to be output;
An image simplification step of simplifying the image to be output by the tactile interface device based on the determined type of the image;
An image display step of outputting the simplified image through the computing device and the tactile interface device; And
An output image adjustment step of enlarging, reducing or moving the image output through the tactile interface device by the user's input; Including,
The image display step,
An output area determining step of setting an area to be output through the tactile interface device among the images as an output area;
A first display information generating step of generating first display information including information of the image to be displayed on a computing device based on the image; And
A second display information generating step of generating second display information including image information of an output region of a simplified image to be tactilely displayed in a tactile interface device based on the image; Including,
The tactile display displayed by the image output step,
A content area tactilely displaying image information of the output area of the simplified image; And
A location information area tactilely displaying information on the output area; Including,
The image simplification step,
A grayscale conversion step of converting the image to grayscale to generate a grayscale image,
The second display information generation step,
An image viewer generating second display information by determining the step of rising or falling of each of the pins by mapping information of the output area of the grayscale image to the coordinates of the pin in a plurality of pins of the tactile interface device How to provide functionality.
The method according to claim 1,
The image selection step,
An image database list display step of displaying a list of one or more image databases in which the images are stored;
An image database selection step of selecting an image database from which an image is to be loaded by a user input;
An image category display step of displaying a category of images stored in the image database;
An image category selection step of selecting a category by user input;
An image search step of searching for an image by user input;
An image list display step of displaying a list of printable images based on at least one of the selected image database, the selected category, and the image search results; And
A display image selection step of selecting an image to be output by a user input; How to provide an image viewer function, including.
delete delete delete The method according to claim 1,
The output image adjustment step,
An output image enlargement / reduction step of enlarging or reducing the image tactilely displayed by the tactile interface device and outputting the image; And
An output image moving step of moving and outputting the position of the image tactilely displayed by the tactile interface device; How to provide an image viewer function, including.
delete delete delete A device implemented as a computing device including a processor and connected to the computing device to provide an image viewer function through a tactile interface device capable of interacting with a user,
The computing device includes an image viewer module; Including,
The image viewer module,
An image selection unit for selecting an image to be output by the user's input; And
An image output unit for outputting the selected image through the computing device and the tactile interface device; Including,
The image output unit,
An image discrimination step of determining the type of the image based on the information of the image to be output;
An image simplification step of simplifying the image to be output by the tactile interface device based on the determined type of the image;
An image display step of outputting the simplified image through the computing device and the tactile interface device; And
An output image adjustment step of enlarging, reducing or moving the image output through the tactile interface device by the user's input; And
The image display step,
An output area determining step of setting an area to be output through the tactile interface device among the images as an output area;
A first display information generating step of generating first display information including information of the image to be displayed on a computing device based on the image; And
A second display information generating step of generating second display information including image information of an output region of a simplified image to be tactilely displayed in a tactile interface device based on the image; Including,
The tactile display displayed by the image output unit,
A content area tactilely displaying image information of the output area of the simplified image; And
A location information area tactilely displaying information on the output area; Including,
The image simplification step,
A grayscale conversion step of converting the image to grayscale to generate a grayscale image,
The second display information generation step,
An image viewer generating second display information by determining the step of rising or falling of each of the pins by mapping the information of the output area of the grayscale image to the coordinates of the pin in a plurality of pins of the tactile interface device A device that provides functionality.

Images