JP2009204710A - Tactile presentation apparatus and tactile presentation method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To convert information into braille enabling a visually impaired person and the like to tactilely recognize entries with ease, and to display the braille. <P>SOLUTION: A tactile presentation apparatus for presenting the object data displayed on a screen by means of a tactile display having tactile members arranged in a plurality of arrays includes: an acquisition means acquiring display area information including at least the number and arrangement intervals of the tactile members constituting the tactile display; a character data extraction means extracting character data from the object data; a braille conversion means converting the character data into braille; a layout data extraction means extracting layout data from the object data; a braille layout data creating means creating, on the basis of the layout data, braille strings laid out to display the converted braille by means of the tactile display; and a device information output means outputting device information for presenting the display strings by means of the tactile display. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a tactile sense presentation device and a tactile sense presentation method, and more particularly to a tactile sense presentation device and a tactile sense presentation method for a visually handicapped person or the like to convert and present items in Braille that are easy to tactilely recognize.

  Conventionally, as a method for presenting screen information to a tactile display (tactile presentation means), a tactile pin (tactile member) at a corresponding location is driven up and down by converting the display screen into binary values of white and black. There is a method of presenting information (first conventional example) (for example, see Non-Patent Document 1). Note that the technique disclosed in Non-Patent Document 1 intends to present images and figures in a tactile manner by, for example, performing threshold control and image processing when binarizing.

  Also, as a drawing tool specialized for tactile displays and point plotters, there is a tool that directly outputs braille, graphs, figures, etc. drawn on the screen as a point diagram (second conventional example) (for example, non-patent Reference 2).

  In addition, as a method for confirming a document layout using a tactile display, there is a method for presenting only layout data using dots on a tactile display and confirming text contents by speech synthesis (third conventional example) (for example, , See Patent Document 1).

Furthermore, as a Braille presentation method for character text information, for example, a Braille printer capable of printing a plurality of lines and various types of Braille displays mainly corresponding to a one-line display have been commercialized (fourth conventional technique). Example) (For example, refer nonpatent literature 3 and 4.).
KGS "Dot View (registered trademark)" (http://www.kgs-jpn.co.jp/b_dv2.html) Edel (http://www7a.biglobe.ne.jp/~EDEL-plus/) JP-A-10-232600 Kaneko et al., "Improvement of Braille printer specialized for graphic output", "31st Sensation Agency Symposium", December 2005. KGS "Brail Memo (registered trademark)" (http://www.kgs-jpn.co.jp/piezo.html)

  By the way, information that is difficult for a visually handicapped person or the like to understand by acquiring only one-dimensional information by voice or Braille needs a means that can be grasped two-dimensionally by using tactile sense or the like. On the other hand, in information acquisition environments such as data broadcasting and the Web, GUI (Graphical User Interface) screens and the like described in applications and markup languages are mainstream, and visually impaired people are on the screen as well as sighted people. There is a growing need to operate after understanding the layout. Therefore, in such a background, it is necessary to output characters, graphs, figures, and the like using the components of the GUI screen as dot diagrams even in the tactile display device.

  However, in the binarization process which is the first conventional example described above, a certain amount of graphics and graphs can be displayed by using image processing. Since it is different from the resolution, there is a problem that it is not suitable for general-purpose braille display because it is necessary to convert the number of pixels.

  In addition, the drawing tool of the second conventional example described above can freely arrange figures and Braille, but is not applicable to screen display of applications or display of data described in a markup language. There was a problem that dot plotters and tactile displays were limited.

  Further, the above-described third conventional example does not have a mechanism itself for generating a two-dimensional layout using Braille.

  Furthermore, although the Braille printer as the fourth conventional example described above can present information by arranging braille two-dimensionally, it is not easy to update and solves the difference in the character string length between Braille and ink characters. Have difficulty. The Braille display is easy to update, but there is a problem that it is difficult to present two-dimensional information.

  The present invention has been made in view of the above-described problems, and provides a tactile presentation device and a tactile presentation method for visually impaired persons and the like to convert and present items in Braille that are easy to tactilely recognize. Objective.

  In order to solve the above problems, the present invention employs means for solving the problems having the following characteristics.

  The invention described in claim 1 is a tactile presentation device for presenting target data displayed on a screen by a tactile presentation unit having a plurality of tactile members arranged. Acquisition means for acquiring display area information including at least the number of knowledge members and arrangement intervals, character data extraction means for extracting character data from target data displayed on the screen, and character data obtained by the character data extraction means The Braille conversion means for converting the image data into Braille, the layout data extraction means for extracting layout data from the target data, and the Braille characters converted by the Braille conversion means based on the layout data obtained by the layout data extraction means Braille layout data generation for generating a braille string laid out for display on the tactile sense presentation means Means, device information output means for outputting device information for presenting the braille string generated by the braille layout generating means by the tactile sense presenting means, display area information and user finger position information regarding the tactile sense presenting means Device information acquisition means for acquiring, based on the acquisition information obtained by the device information acquisition means, braille display area control means for controlling the length of the braille string of each item in preference to the layout data, The Braille layout data generation means adjusts the length of the Braille string of each item based on the Braille string area information generated by the Braille display area control means.

  According to the first aspect of the present invention, it is possible for a visually handicapped person or the like to convert and present an item into Braille that is easily tactilely recognized. Further, the same character data can be presented in correspondence with substantially the same position of each item displayed on the display screen.

  According to the second aspect of the present invention, the Braille layout data generating means omits that a part of character data is omitted when Braille cannot be displayed in the area obtained by the Braille display area control means. A sign is added.

  According to invention of Claim 2, the presence or absence of omission can be determined easily.

  The invention described in claim 3 is characterized in that the braille layout data generating means preferentially displays the character data at the user's finger position in each item of the character data without being restricted by the area. .

  According to the third aspect of the present invention, it is possible to display the continuation of the omitted Braille data or all character data.

  The invention described in claim 4 is a tactile sense providing method for presenting target data displayed on a screen by a tactile sense presenting means having a plurality of tactile members arranged. An acquisition step of acquiring display area information including at least the number of knowledge members and an arrangement interval, a character data extraction step of extracting character data from target data displayed on the screen, and character data obtained by the character data extraction step A Braille conversion step for converting the image data into Braille, a layout data extraction step for extracting layout data from the target data, and a Braille character converted by the Braille conversion step based on the layout data obtained by the layout data extraction step. A braille string laid out for display on the tactile sense presentation means is generated. A braille layout data generation step, a device information output step for outputting device information for presenting the braille string generated by the braille layout generation step by the tactile sense presenting means, and an acquisition obtained by the device information acquisition step A braille display area control step for controlling the length of the braille string of each item with priority given to the layout data based on the information, and the braille layout data generation step is generated by the braille display area control step. The length of the Braille string of each item is adjusted based on the Braille string area information.

  According to the fourth aspect of the present invention, the visually impaired person or the like can convert and present the item into Braille that is easily tactilely recognized. Further, the same character data can be presented in correspondence with substantially the same position of each item displayed on the display screen.

  According to the present invention, it is possible for a visually handicapped person or the like to convert and present an item into Braille that is easily tactilely recognized.

<Outline of the present invention>
The present invention corresponds to various information resources obtained from data broadcasting, Web, etc., for example, visually impaired persons such as blindness, amblyopia, deafblindness, etc. convert the items into Braille that is easy to tactilely recognize, and a tactile display It can be grasped by tactile sense presentation means such as. That is, according to the present invention, information on a document or GUI screen is presented in braille in accordance with a display area at an arbitrary position on the tactile display, and a user such as a visually impaired user can determine the positional relationship in the document and the item contents of the GUI screen Provides a method of obtaining a two-dimensional image on a tactile display with a list.

  Specifically, according to the present invention, in a method of presenting screen information such as a GUI to a visually impaired person, character information displayed in various parts of the screen is presented in braille on a tactile display, and the text information of each item and the screen information are displayed. Presents layout data such as the position.

  In the present invention, in a display format expressed in a markup language format such as data broadcasting or Web, character information is converted into a tactilely readable Braille string and variously reflected in a screen layout. Present on a tactile display with various display areas.

  That is, in order to realize the above-described contents, the present invention is based on a condition in which tags included in a parser or a parser (XML (Extensible Markup Language) or the like) for extracting character text and layout data are set in advance. Analysis and display), and a function for setting a tactile-readable Braille output method, etc., and optimizing the braille display according to the layout data and the braille display area of the tactile display .

  Hereinafter, embodiments in which the tactile sensation presentation apparatus and the tactile sensation presentation method according to the present invention having the above-described features are preferably described in detail with reference to the drawings.

<First Embodiment>
<Outline configuration example of tactile presentation device>
FIG. 1 is a diagram illustrating an example of a schematic configuration of a tactile sense presentation device according to the present invention. 1 is implemented by a general-purpose computer such as a PC (Personal Computer).

  1 includes a character data extraction unit 11, a Braille conversion unit 12, a layout data extraction unit 13, a Braille layout data generation unit 14, a device information output unit 15, a tactile display (tactile presentation). Means) 16, device information acquisition means 17, Braille display area control means 18, browser (playback means) 19, and display output means 20.

  The character data extraction unit 11 inputs information resources 21 which are display data (GUI screen data, etc.) and are described in HTML (HyperText Markup Language), XML, etc., and character text information from the information contained in the information resources 21 To extract. Note that the character data extraction unit 11 can convert the entire image included in the information resource 21, for example, an image, a document, or the like into digital still image information using an image scanner or the like when there is a character in the image included in the information resource 21. Character information can be acquired by performing optical character recognition processing using OCR (Optical Character Recognition). Further, the character data extracting unit 11 outputs the extracted character text information to the Braille converting unit 12.

  The braille conversion means 12 associates the text data of the character text information (such as ink characters) mixed with the kanji kana extracted by the character data extraction means 11 with “Hiragana”, and further converts the hiragana data into Braille data. The Braille conversion unit 12 outputs the converted Braille data to the Braille layout data generation unit 14.

  The layout data extraction unit 13 inputs the information resource 21 of the display data described above, and extracts layout data included in the information resource 21. In addition, layout data can set the layout position from information such as the screen size, position and size included in the tag and actual data, and further read the entire image on the information resource by the above-described scanner, The layout position can be extracted from the read still image information.

  In this way, by using the layout data extraction means 13, it is possible to acquire the positional relationship on the screen of character data such as where the character information is described on the screen. In addition, the layout data extraction unit 13 outputs the extracted layout data to the braille layout data generation unit 14.

  The Braille layout data generation means 14 is the Braille data obtained by the Braille conversion means 12, the layout data obtained by the layout data extraction means 13, and the finger actually used by the user obtained by the Braille display area control means 18 described later. Based on the position information and the like, a final braille display layout of items (objects) displayed by integrating these data is generated.

  In other words, the Braille layout data generation unit 14 converts the coordinate system of the item position and size according to the coordinate system of the display area of the tactile display 16, and adjusts the length of the Braille string of each item based on this coordinate system. Then, each object is arranged on the tactile display 16.

  At that time, the Braille layout data generation unit 14 draws the figure at a designated place when a figure drawing such as a square or a circle is designated in advance, and when there is no designation in advance, the Braille layout data generation means 14 draws the Braille within the object area. Arrange character boxes as much as possible. Thereby, the character information can be presented in braille on the tactile display with the same layout as the screen.

  Further, the Braille layout data generation means 14 has functions such as line breaks in units of division writing, setting of braille line intervals, page management, and the like. Furthermore, the Braille layout data generation means 14 can switch between a display that prioritizes the layout on the screen and a display that prioritizes the text information by changing the layout, for example, according to a user's preset setting. As a result, when priority is given to the layout on the screen, information such as the actual arrangement of items on the screen can be presented on the tactile display so that the user can grasp it. In addition, when priority is given to text information, the continuation of the omitted Braille data or all character data can be displayed.

  Further, the Braille layout data generation unit 14 outputs the generated Braille display layout to the device information output unit 15.

  The device information output means 15 is provided with tactile pins (tactile members) that can move to the unevenness arranged on the tactile display 16 in order to output the data acquired by the braille layout data generation means 14 to the tactile display 16. Output information to the device such as which pin to project is generated. The device information output unit 15 displays the generated output content using the tactile display 16.

  The tactile display 16 has, for example, a plurality of tactile pins that can move up and down in a vertical n (book) and a horizontal m (book). The tactile display 16 projects the corresponding tactile pins based on the presentation information obtained by the device information output means 15 and presents data by the unevenness of the tactile pins.

  The device information acquisition unit 17 acquires display area information of the tactile display 16, finger position information, instruction information from the user using the tactile display 16, and the like. Specifically, the device information acquisition unit 17 determines the number of tactile pins of vertical n (books) and horizontal m (books), the pin diameter, the distance between the pins, the number of brailles per line, and 1 degree. Information such as a braille display area matrix is acquired as the number of lines that can be presented.

  Further, the device information acquisition unit 17 acquires information on which position (on which pin) the finger is placed. As a method for detecting the finger position, a general method can be used. For example, one or a plurality of cameras or the like are used to detect the finger position, and the finger position is touched by a change in the reflection pattern due to the vertical movement of the finger. An operation (operation in which the finger is once removed from the touch surface of the pin and placed again) is detected. In addition to this, there is a method of detecting the pressure with a finger on the tactile pin using a sensor or the like, but the present invention is not limited thereto. Further, as the information on the designated position, for example, coordinate information on the screen can be used.

  Further, the device information acquisition unit 17 acquires instruction information from the user by “determination”, “return”, “display switching”, and the like provided on the tactile display 16. The device information acquisition unit 17 outputs the acquired display area information, finger position information, instruction information, and the like to the Braille display area control unit 18.

  The braille display area control means 18 calculates a braille display area that can be presented on the tactile display 16 based on various information obtained by the device information acquisition means 17. The braille display area control means 18 outputs information on the display area such as the position and size that can be displayed as braille on the tactile display 16 to the braille layout data generation means 14.

  Specifically, the braille display area control means 18 uses, for example, “character spacing (dots), line spacing (dots)” and “number of characters (characters / lines), number of lines (lines / pages)” as control parameters. Various controls such as conversion control, display control of data and Braille are performed.

  The browser 19 is a reproduction means for inputting display data of the information resource 21 and causing the display output means 20 such as a display to reproduce and output it on the screen. That is, the browser 19 represents general software that displays information on the computer for a certain purpose and provides browsing. Further, the browser 19 outputs the reproduction information to the display output unit 20. Further, the display output means 20 outputs the reproduction information obtained by the browser 19 to the screen using a GUI or the like.

  The functions of the browser 19 and the display output means 20 described above are functions for determining whether or not the content displayed on the tactile display 16 is actually displayed on a screen such as a display. It may not be provided in the apparatus 10 and may be provided outside.

  In the above-described embodiment, the tactile display 16 can be an existing tactile display device or the like. For example, a relatively inexpensive optical touch panel can be used to directly operate a finger of a user on the tactile display. be able to. Specifically, for example, as a touch panel-mounted tactile display, a tactile display having a display area of 96 × 64 dots (pin spacing 2.4 mm) (for example, manufactured by KGS) and an optical touch panel (Xiroku) And the like mounted on the board can be used. Since this touch panel is a triangulation method using sensors on both sides of the upper part, it does not hinder the touch surface of the tactile display and does not require a marker or the like.

  Here, when a GUI screen is presented by a tactile display or the like, first, menu selection items and the like are converted into objects that are easy to touch and are arranged. Next, this tactile display display screen is binarized and presented by upper and lower tactile pins. The presented GUI operations can be key operations (decision, return, display switching, etc.) from the tactile display and mouse operations (pointing and clicking (touch operations)) using the touch panel. When an object is selected, the corresponding text information is synthesized by voice and fed back.

  In addition to displaying on the tactile display, the display may be performed using another tactile presentation device, or may be displayed on a display of a general-purpose computer such as a PC.

  Here, in the case of supporting a general-purpose computer such as a PC, the resolution of the screen (1024 × 768) and the resolution of the tactile display (96 × 64) are different, so that the entire PC screen is displayed on the tactile display. Pixel number conversion is performed.

  Further, the optimum value is adjusted by the calibration function of the touch panel so that the finger position of the user on the tactile display matches the position indicated by the mouse cursor on the PC screen.

  Thereby, when a finger is moved on the tactile display, the mouse cursor on the PC screen can follow. If the mouse cursor is positioned accurately on the object on the PC screen at the moment when the object is touched with a finger, it can be selected correctly.

  As described above, according to the present embodiment, the character text information displayed on the GUI can be displayed in Braille according to the display area of the tactile display while reflecting the layout data. Therefore, visually impaired persons can convert the items into Braille that can be easily tactilely recognized and present them.

<Tactile sense presentation device 10: hardware configuration example>
Here, the haptic presentation device 10 described above includes a CPU (Central Processing Unit), a volatile storage medium such as a RAM (Random Access Memory), a non-volatile storage medium such as a ROM (Read Only Memory), a mouse and a keyboard, It can be configured by a computer including an input device such as a pointing device, a display unit for displaying images and data, and an interface for communicating with the outside.

  Therefore, each component function included in the tactile sense presentation device 10 can be realized by causing the CPU to execute a program describing these functions. These programs can also be stored and distributed in a recording medium such as a magnetic disk (floppy disk, hard disk, etc.), optical disk (CD-ROM, DVD, etc.), semiconductor memory, or the like.

  That is, an execution program for causing a computer to execute the processing in each configuration described above is generated, and the tactile presentation processing can be realized by installing the program in a general-purpose personal computer such as a PC or a server, for example. .

  Here, a hardware configuration example of a computer that can execute the program according to the present invention will be described with reference to the drawings. FIG. 2 is a diagram illustrating an example of a hardware configuration capable of realizing the tactile sense presentation process according to the present invention.

  The computer main body in FIG. 2 includes an input device 31, an output device 32, a drive device 33, an auxiliary storage device 34, a memory device 35, a CPU 36 for performing various controls, and a network connection device 37. These are connected to each other by a system bus B.

  The input device 31 includes a keyboard and a pointing device such as a mouse operated by a user, a voice input device such as a microphone, and the like, and inputs various operation signals such as execution of a program from the user.

  The output device 32 includes a display for displaying various windows and data necessary for operating the computer main body for performing processing in the present invention, a speaker for outputting sound, and the like. Execution progress, results, etc. can be displayed or voice output. In this embodiment, tactile sense presenting means is provided. At this time, the above-described dedicated unit or the like can be externally attached.

  The input device 31 and the output device 32 may be integrated input / output means such as a touch panel, for example. In this case, a predetermined position using a user's finger or a pen-type input device is used. You can input by touching.

  Here, in the present invention, the execution program installed in the computer main body is provided by the recording medium 38 such as an optical disk, for example. The recording medium 38 on which the program is recorded can be set in the drive device 33, and the execution program included in the recording medium 38 is installed in the auxiliary storage device 34 from the recording medium 38 via the drive device 33.

  The auxiliary storage device 34 is a storage means such as a hard disk, and can store an execution program in the present invention, a control program provided in a computer, and the like, and can perform input / output as necessary.

  The memory device 35 stores an execution program read from the auxiliary storage device 34 by the CPU 36. Note that the memory device 35 includes a RAM, a ROM, and the like.

  The CPU 36 controls processing of the entire computer, such as various operations and input / output of data with each hardware component, based on a control program such as an OS (Operating System) and an execution program stored in the memory device 35. Each processing can be realized. Further, the CPU 36 can acquire various kinds of information necessary during execution of the program from the auxiliary storage device 34, and the CPU 36 can also store processing results and the like.

  The network connection device 37 obtains an execution program from another terminal connected to the communication network by connecting to a communication network or the like, or an execution result obtained by executing the program or the present invention. The execution program itself can be provided to other terminals. Further, the network connection device 37 outputs tactile presentation contents, text data, voice data, etc. to a tactile display connected in a state where data can be transmitted and received via a communication network, or input by a user key operation on the tactile display or the like. Information can be obtained.

  With the hardware configuration as described above, a special device configuration is not required, and a tactile sense presentation process can be realized efficiently at a low cost. In addition, by installing the program, it is possible to easily acquire the tactile sense presentation process in the present embodiment.

<Tactile presentation method in the tactile presentation device 10>
Next, the tactile sense presentation method in the present embodiment will be described with reference to the drawings. FIG. 3 is a diagram illustrating an example of a GUI screen according to the first embodiment. FIG. 3A is a diagram illustrating an example of the layout of the GUI screen, and FIG. 3B is a diagram illustrating an example of a markup language (for example, HTML) for the GUI screen.

  In the tactile sense presentation method according to the present embodiment, for example, when a GUI screen as shown in FIG. 3A is displayed on the tactile display, the character data extraction unit 11 first uses the markup language shown in FIG. Extract data. In this case, for example, data of a predetermined tag (for example, text) in the markup language shown in FIG. 3B is extracted. Next, the extracted character data is converted into hiragana for conversion into Braille by the Braille conversion unit 12.

  That is, the “data broadcasting selection menu”, “news”, “weather information”, “sports”, “TV program guide”, “earthquake information”, “typhoon information”, “program” shown in FIG. "Linked", etc., are described as "Deta Hoso Sensaku Me New", "News", "Kisho Jyoho", "Sports", "Telebi Bangumi Hyo", respectively. -"," Jijinjoho "," Taifujohoho "," Bangumi Rendo ". Further, the Braille conversion unit 12 generates a Braille string corresponding to the hiragana of each item of the GUI. Further, the layout data extracting means 13 extracts the position information of each character data item shown in FIG.

  Next, the Braille layout data generation unit 14 converts the Braille strings of each item described above into the coordinate system calculated by the Braille display area control unit 18 based on the information acquired by the layout data extraction unit 13 and arranges them.

  Here, FIG. 4 is a diagram illustrating an example of presentation contents of Braille corresponding to FIG. In addition, the black circle part of the tactile pin shown in FIGS. 4B and 4D shows a protruding part of the tactile pin. Moreover, the braille corresponding to the hiragana shown in FIG. 4A is shown in FIG. 4B, and the Braille corresponding to the hiragana shown in FIG. 4C is shown in FIG. Furthermore, although the display area of the character data in the tactile display shown in FIGS. 4A to 4D is displayed as a rectangle for convenience, the area is not actually displayed.

  Here, since braille basically corresponds to hiragana, the number of characters increases when they are written in kanji and the like, and not all can be displayed due to the restriction of the corresponding display area. Therefore, when all braille strings cannot be displayed in the layout frame of each item, the length of the character string of each item is adjusted as shown in FIG. 4A and an ellipsis (abbreviated braille) 41 is added. . As a result, as shown in FIG. 4B, it is possible to present the braille on the GUI screen with priority given to the layout data of the screen.

  Here, in this embodiment, there is a mechanism for displaying characters omitted by the ellipsis 41. Specifically, the Braille character string omitted by giving priority to the layout data indicates the position of the finger 42 that designates a predetermined part of the tactile display as shown in FIG. Obtained from the information obtaining means 17. Further, using the acquired finger position information, only the character strings in the items are displayed in the layout frame of the coordinate system newly calculated by the braille display area control means 18 as shown in FIG. Thus, the corresponding braille is presented as shown in FIG. 4D, and the user or the like can confirm the display by giving priority to the character information.

<Second Embodiment>
Next, a second embodiment will be described. The second embodiment is an example showing a display method when layout data to be displayed on the tactile display is given in CSS (Cascading Style Sheets) in the above-described first example.

  Here, CSS is a specification that specifies how to modify (display) HTML and XML elements, and is one of the recommendations by the W3C, realizing the philosophy of separating the structure and appearance of documents. This is one of the specific specifications of the style sheet that was proposed to do this.

  CSS has new design functions while incorporating elements that can realize most of designs that can be expressed in HTML. The CSS can change the display for each medium by switching the style sheet according to the medium for displaying the page. In addition, CSS can superimpose (cascade) the effects of CSS defined by user agents (in many cases, web browsers), website creators, and users.

  Therefore, in the second embodiment, for example, an original XML file created by the browser at the time of screen rendering and a CSS file for tactile display display are acquired, and the object specified by CSS is coordinate-transformed and rendered. When the object includes text information, it can be displayed in Braille within the drawing range at the designated location. The functional configuration of the second embodiment is the same as that of the first embodiment described above, and a description thereof is omitted here.

  FIG. 5 is a diagram illustrating an example of a GUI screen according to the second embodiment. FIG. 5A is a diagram illustrating an example of the layout of the GUI screen, and FIG. 5B is a diagram illustrating an example of the CSS for the GUI screen.

  When the GUI screen shown in FIG. 5A is displayed on the tactile display, the braille layout data generating means 14 uses the CSS data (for example, “item” tag for displaying the tactile display shown in FIG. 5B). Information).

  Here, FIG. 6 is a diagram illustrating an example of presentation contents of braille corresponding to FIG. In addition, the braille corresponding to the hiragana shown in FIG. 6A is shown in FIG. Moreover, although the display area of the character data in the tactile display shown in FIGS. 6A and 6B is displayed as a rectangle for convenience, the area is not actually displayed.

  Here, based on the information acquired by the layout data extraction means 13, it is converted into the coordinate system calculated by the Braille display area control means 18, and arranged for each item as shown in FIG. Presentation by tactile sensation is performed by the unevenness of the tactile pins shown in b). At this time, if not all braille strings can be displayed within the layout frame of each item, switching between the braille display on the GUI screen giving priority to the layout data and the display giving priority to the character information is switched as in the first embodiment described above. Thus, it is possible to make the user confirm.

  It should be noted that the display giving priority to the layout on the screen as shown in FIG. 4 and the display giving priority to the text information by changing the layout as shown in FIG. 6 can be arbitrarily switched according to a preset user designation. Can do.

<Tactile presentation processing procedure>
Here, an example of the tactile sense presentation processing procedure will be described with reference to a flowchart. FIG. 7 is a flowchart illustrating an example of a tactile sense presentation process procedure according to this embodiment. In the flowchart shown in FIG. 7, first, an information resource is input (S01), and hiragana conversion and Braille conversion of text information are performed on the input information resource (S02).

  Next, a braille layout for displaying layout data with priority is generated (S03). Here, it is determined whether or not there is an abbreviated braille indicating that the entire text cannot be displayed due to the limitation of the display area and the like, only the characters that fall in the partial area are displayed, and thereafter omitted (S04). If there is an abbreviated Braille (YES in S04), it is next determined whether or not there is a character display operation by the user (S05). When there is a character display operation by the user (YES in S05), the user's finger position information is acquired (S06), and a Braille layout (character information priority) of all character data corresponding to that portion is generated (S07). .

  Next, when there is no abbreviated Braille (NO in S04), or when there is no character display operation by the user (NO in S05), the process is further output to a device such as a tactile display after the processing of S07 is completed (S08). .

  Next, it is determined whether or not another information resource is input (S09). If there is a resource input, the process returns to the above-described processing of S01, and another information resource is input to perform subsequent processing. I do. Also. If no other information resource is input in the process of S09, the process ends.

  Through the above-described tactile sense presentation process, visually impaired persons or the like can convert and present items in Braille that are easy to tactilely recognize. Further, by generating an execution program for causing a computer to execute the processing in each configuration described above, and installing the program in a general-purpose personal computer such as a PC or a server, for example, the tactile sense presentation processing can be realized. .

  In the present embodiment, it is possible to acquire character data from a browser and use it as a vibration pattern to present vibration to the vibrator of the tactile display according to a command or the like. Further, it is possible to acquire a key operation command from the tactile display and deliver a command other than the control of this interface in response to a request from the browser.

  Moreover, the display area of the current tactile display is smaller than the display area of the screen of a television or a personal computer. Also, in general, the braille notation has a larger number of characters than the inscription in kanji mixed with kanji. Therefore, for example, as shown in the second embodiment described above, by using XML and CSS, each item can be presented in braille with as many characters as possible while reflecting the layout data. Similarly, it is possible to switch to a display in which text information is prioritized or to make a presentation according to the display area of the tactile display to be used by switching the CSS, for example.

  As described above, according to the present invention, the visually impaired person or the like can convert and present the item into Braille that is easily tactilely recognized. Specifically, according to the Braille display method of the tactile display of the present invention, information on a document or GUI screen can be presented in Braille according to the display area at any location on the tactile display, and visually impaired persons, etc. However, it is possible to two-dimensionally grasp and acquire the positional relationship in the document and the item contents of the GUI screen with a list on the tactile display.

  Although the preferred embodiment of the present invention has been described in detail above, the present invention is not limited to the specific embodiment, and various modifications, within the scope of the gist of the present invention described in the claims, It can be changed.

It is a figure which shows an example of schematic structure of the tactile sense presentation apparatus in this invention. It is a figure which shows an example of the hardware constitutions which can implement | achieve the tactile sense presentation process in this invention. It is a figure which shows an example of the GUI screen in 1st Embodiment. It is a figure which shows an example of the presentation content of the Braille corresponding to FIG. It is a figure which shows an example of the GUI screen in 2nd Embodiment. It is a figure which shows an example of the presentation content of the braille corresponding to FIG. It is a flowchart which shows an example of the tactile sense presentation process procedure in this embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Tactile sense presentation apparatus 11 Character data extraction means 12 Braille conversion part 13 Layout data extraction part 14 Braille layout data generation part 15 Device information output means 16 Tactile display (tactile sense presentation means)
17 Device information acquisition means 18 Braille display area control means 19 Browser (reproduction means)
20 display output means 21 information resource 31 input device 32 output device 33 drive device 34 auxiliary storage device 35 memory device 36 CPU
37 Network connection device 38 Recording medium 41 Ellipsis (abbreviated Braille)
42 fingers

Claims (4)

In a tactile sense presentation device for presenting target data displayed on a screen by a tactile sense presenting means having a plurality of tactile members arranged,
Obtaining means for obtaining display area information including at least the number of tactile members constituting the tactile sense presenting means and an arrangement interval;
Character data extraction means for extracting character data from the target data displayed on the screen;
Braille conversion means for converting character data obtained by the character data extraction means into Braille;
Layout data extraction means for extracting layout data from the target data;
Braille layout data generating means for generating a braille string laid out to display the braille converted by the Braille converting means on the tactile sense presenting means based on the layout data obtained by the layout data extracting means;
Device information output means for outputting device information for presenting the braille string generated by the braille layout generating means by the tactile sense presenting means;
Device information acquisition means for acquiring display area information and user finger position information related to the tactile sense presentation means;
Braille display area control means for controlling the length of the Braille string of each item preferentially based on the acquisition information obtained by the device information acquisition means,
The tactile sensation presentation apparatus, wherein the braille layout data generation means adjusts the length of the braille string of each item based on the braille string area information generated by the braille display area control means. The Braille layout data generating means
The tactile sensation presentation apparatus according to claim 1, wherein when the braille cannot be displayed in the area obtained by the braille display area control means, an abbreviation code indicating that a part of the character data is omitted is added. . The Braille layout data generating means
3. The tactile sensation presentation apparatus according to claim 1, wherein, in each item of the character data, the character data at the user's finger position is preferentially displayed without being limited by a region. In a tactile sense presentation method for presenting target data displayed on a screen by a tactile sense presenting means having a plurality of tactile members arranged,
An acquisition step of acquiring display area information including at least the number of tactile members constituting the tactile sense presenting means and an arrangement interval;
A character data extraction step of extracting character data from the target data displayed on the screen;
Braille conversion step of converting the character data obtained by the character data extraction step into Braille,
A layout data extraction step for extracting layout data from the target data;
Based on the layout data obtained by the layout data extraction step, a Braille layout data generation step for generating a Braille string laid out to display the Braille converted by the Braille conversion step on the tactile sense presenting means;
A device information output step for outputting device information for presenting the braille string generated by the braille layout generation step by the tactile sense presenting means;
A braille display area control step for controlling the length of the braille string of each item preferentially based on the acquisition information obtained by the device information acquisition step;
The tactile layout data generation step adjusts the length of the braille string of each item based on the braille string area information generated by the braille display area control step.

Images