JPH09258946A - Information processor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enable even handicapped person in eyesight to execute the selection operation of a window and an icon by cutputting information such as the icon and the like in a multi-window system with sound and Braille. SOLUTION: Screen layout information transmitted from an application 2 to an operating system 3 is hooked by a screen layout information preservation means 9 in a sound/Braille output means 7 and it is stored in a buffer for screen layout information preservation 10. A screen layout information read means reads screen layout information of the buffer 10 whenever an output operation part 6 detects the depression of a control key, and transmits it to a screen layout information analysis means 13. The analysis means 13 analyzes screen layout information and outputs a data string. A voice/Braille data conversion/buzzer control part 11 converts the data string into voice/Braille, outputs it to a speaker/sound synthesis device/Braille display device 12 and informs a user of a window, the name of the icon, a coordinate and a distance with voice and Braille.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to means for outputting information such as windows, icon names, and coordinates in a multi-window system of an information processing apparatus such as a word processor and a computer by voice or Braille.

In recent years, information processing devices such as personal computers and word processors have come to be offered at low prices, and have come to be widely used not only in business in companies but also at the individual level.

Moreover, the conventional character-based O
The S system is in a state of rapid transition to a graphic-based multi-window system. However, for the visually impaired, it is the current situation that the conventional character-based system which can be used for voice and braille cannot be used at all when it is changed to the multi-window system.

Therefore, in order for the visually impaired person to be able to use the information processing apparatus, it is required to output information such as windows and icons in voice or Braille in the multi-window system.

[0005]

2. Description of the Related Art FIG. 6 is a diagram for explaining a conventional information processing apparatus, FIG. 6 (a) shows a case of a character-based information processing apparatus, and FIG. 6 (b) shows a graphic-based information processing apparatus. The case of a processing device is shown.

First, in the character-based information processing apparatus shown in FIG. 6 (a), when a character output request is issued from the application 2, the screen output means 4 of the operating system (OS) section 3 causes a predetermined character code. Is output, the character code is stored in the character VRAM 50, and the character code is converted into a predetermined character pattern and displayed on the display 5.

At this time, the character codes in the character VRAM 50 are sequentially converted into voice and Braille data, and the voice is output from a speaker via a voice synthesizer or the like, or Braille is displayed on the Braille display device.

Next, in the graphic-based information processing apparatus shown in FIG. 6B, when a character output request is issued from the application 2, the operating system (OS) unit 3
The screen output means 4 converts the predetermined character code into graphic data and outputs the graphic data, which is stored in the bitmap memory in the display 5 and displayed on the display 5.

At this time, a predetermined character code is output from the name of the window or icon generated by the screen output means 4 of the operating system (OS) section 3 from the window or icon designated by the operator using the map cursor,
It is converted into voice and Braille data, and is output by a speaker through a voice synthesizer or the like, or Braille is displayed on a Braille display device.

That is, in the conventional information processing apparatus,
There have been means for outputting a text file as voice, and means for outputting the names of windows, icons, etc. in which the mouse cursor is positioned, as voice.

However, these means are created for the purpose of assisting the operation of a sighted person who grasps the information of the entire screen and moves the mouse cursor to the position of a predetermined window or icon for selection. It was

[0012]

Therefore, since the conventional information processing apparatus has only the function of outputting the icon or the like at the position pointed by the mouse cursor, the information of the entire screen can be grasped and the system operation is performed for the first time. It was impossible for a visually impaired person who could not grasp the information to operate the mouse cursor to select the target window or icon.

That is, the information of windows and icons on the display screen, for example, names, coordinates, etc., is not outputted to the visually impaired by voice output or Braille output. As a technique for outputting information on a similar display screen by voice, a technique of “outputting voice when the position of the cursor enters or leaves an arbitrary graphic region” is disclosed.
07-168691 publication "Display graphic area selection method"},
"Only when the cursor moves to the boundary of the object,
"Technology for instructing the next operation by voice" (Japanese Patent Laid-Open No. 06-311649 "Voice guidance device") or "In a character reading device, the recognition result is read out by voice synthesis to improve the correction work of the recognition result." Technology {Japanese Unexamined Patent Publication No. 04-280389, "Character recognition device"} or "Notification as a sound that the cursor position is in an arbitrary window" {Japanese Unexamined Patent Publication No. 05-40589, "Multi-window display method and its Device "} is known, but none of them outputs windows, icon names, coordinates, etc. on the display screen in voice or Braille.

As a technique for changing the sound of a buzzer or the like, a technique of "arranging four speakers so that the operator can auditorily recognize the movement of the cursor" {Japanese Patent Laid-Open No. 07-84724 "Input device" Although the technique is known, it does not inform the operator of the direction and distance of the current cursor position with respect to the target window or icon.

Further, as a technique of changing the shape of the cursor and displaying it on the screen, a technique of "changing the appearance of the cursor as a means for judging whether or not the voice uttered by oneself is currently recognized" is { JP, 06-342365, A "Linear voice control method and device"} or "When the user selects the voice recognition button, the shape of the cursor is changed to the shape of the mouse cursor on the voice recognition button (for example, a microphone pattern). ) Is used to change the shape of the cursor as a means to make the user recognize the operation method by prompting the user to speak, etc. ”(Japanese Patent Laid-Open No. 05-341951,“ Voice input operation device ”, etc. As is known, none of them notifies the operator of the shape of the cursor displayed by the information processing apparatus (for example, the shape for notifying the operator that mouse input is not possible).

Further, as a technique for skipping a cursor displayed on a display screen by a predetermined distance and displaying it, "the entrance point and the exit point of a predetermined area on the display screen are set in advance and the entrance point is set. When the cursor moves to a point, the cursor jumps to the exit point "
-7629 Publication "Pointing cursor jump method"} or "Fine adjustment of the cursor position roughly specified on the touch panel by keyboard and mouse operations, and skipping the cursor to select the desired one" Kaihei 05-108
No. 230 publication "Screen display device"} or "When the cursor reaches a predetermined position, jumping to another icon to eliminate the need for precise alignment" technology {Japanese Patent Laid-Open No. 06-289834 There are "input device and display device"}, etc., but in both cases, the operator recognizes and selects the target window and icon without moving the cursor, and outputs the result as voice or Braille. Therefore, the cursor is not jumped to the position of the selected window or icon and the target window or icon is not selected.

In view of the above-mentioned conventional drawbacks, the present invention provides information processing equipped with means for outputting voice and Braille to enable visually impaired persons to select windows, icons, etc. on a multi-window system. The purpose is to provide a device.

[0018]

FIG. 1 is a block diagram showing the principle of the information processing apparatus of the present invention. The above problems can be solved by an information processing device configured as described below, specifically, a voice / Braille output unit and a related unit.

(1) An information processing apparatus having means (voice / Braille output means) 7 for outputting information such as icons in a multi-window system in voice and Braille, and moving the mouse cursor from the application program 2, A means for hooking the screen layout information output in response to a window / icon output request and saving it in the screen layout information saving buffer (screen layout information saving means) 9, and moving the mouse cursor, mouse, and keyboard.
Means for taking out predetermined screen layout information from the screen layout information storage buffer 10 based on the output request from 1 (screen layout information reading means) 8
And analyzing the retrieved screen layout information,
An output unit 12 for outputting a data string for instructing information such as windows and icons on the screen, and a voice / Braille output unit for converting the data string into voice and Braille and outputting it (voice / Braille data conversion / buzzer control unit). ) Voice consisting of 11 /
It is configured to include Braille output means 7.

(2) In the information processing device according to the above item (1), information such as windows and icons on the screen is displayed on or around a point of interest indicated by the mouse cursor. ， Configure to be the name and coordinates of the icon.

(3) The information processing apparatus according to the above item (1), which detects a moving direction when the point of interest pointed by the mouse cursor moves to the target (processing step in FIG. 3). 106), the information such as windows and icons on the screen may be names, directions, distances, etc. of windows, icons and the like in the direction detected by the moving direction detecting means (processing step 106 in FIG. 3). To configure.

(4) The information processing device according to the above item (3), wherein the scale of the buzzer sound is changed in accordance with the distance between the target point and the target pointed by the mouse cursor, and the sound is output. A means (voice / Braille data conversion / buzzer control unit) 11 is provided.

(5) The information processing apparatus according to the above item (1), wherein means A for detecting a predetermined first key operation is provided, and the predetermined key detected by the first key operation detecting means A is provided. On the basis of the instruction by the first key operation, the window around the point of interest pointed by the mouse cursor, the name of the icon, the coordinates, and the distance are output by voice or Braille.

(6) The information processing apparatus according to the above item (1), wherein means B for detecting a predetermined second key operation is provided, and the predetermined key detected by the second key operation detecting means B is provided. The coordinates of the position of the point of interest pointed by the mouse cursor and the names and coordinates of the window and icon above the point of interest are output in voice or Braille based on the instruction by the second key operation of To configure.

(7) The information processing apparatus according to the above item (1), wherein when the screen layout information analysis means 13 detects a change in the shape of the mouse cursor displayed by the information processing apparatus, the mouse It is configured to output changes in the shape of the cursor in voice or Braille.

(8) The information processing apparatus according to the above item (1), wherein means C for detecting a predetermined third key operation is provided, and the predetermined key detected by the third key operation detecting means C. Based on the third key operation of, the current shape of the mouse cursor displayed by the information processing apparatus is output in voice or Braille.

(9) In the information processing apparatus described in the above item (5), means D for detecting a predetermined fourth key operation is provided to display the voice, the window output in Braille, and the icon. When a mouse is selected by the key operation detected by the third key operation detecting means D, a means for moving the mouse cursor of the point of interest onto the selected window or icon is provided.

That is, the information processing apparatus of the present invention has the configuration shown in FIG. In the figure, 1 is used to indicate a position on the screen with a mouse or keyboard. 2 is an application program. 3 is an OS unit. Outputs of windows, icons, mouse cursors, etc. from the system or application 2 are displayed on the display of 5 by using the screen output means of 4.

7 voice / Braille output means are operated by 6 output operations (depressing operation of a predetermined control key) from an input device such as a keyboard and a mouse 1. In this means, the screen layout information saving means 9 saves the screen layout information in the buffer 10 for saving the screen layout information.
The output operation of 6 causes the screen layout information reading means of 8 to operate, and the layout information at the position designated by the output operation 6 and the like is read from the buffer for storing screen layout information of 10. By the above output operation 6 etc., 11 voice / Braille data conversion / buzzer control units,
Voice / Braille data conversion and buzzer sound control are performed, and sound, Braille, and buzzer sound are output from each of the 12 output devices.

The information processing apparatus of the present invention executes the following operations by each means shown in FIG. 1) Screen layout information to be output by hooking (takeover) requests for movement of the mouse cursor, output of windows, icons, etc. to the OS unit 3 by the system / application 2 (to be described later) As described above, the window and icon position coordinates, the area data for displaying the window and icon, the image data for displaying in the area, the character code, and the cursor code) Save to 10, convert to voice and braille data, and output voice and braille.

2) When the mouse cursor is moved or an output request is made by the user, the screen layout information above the point of interest such as the mouse cursor is taken out from the screen layout information storage buffer 10 and converted into voice and Braille data. Convert and output voice and Braille.

3) The movement of the point of interest such as the mouse cursor is detected in the screen layout information analysis means 13 (by storing the history of the coordinates of the mouse cursor, the movement direction of the mouse cursor can be calculated. Then, the name, direction, distance, etc. of the window, icon, etc. in the moving direction of the point of interest are necessary to analyze the contents of the above-mentioned screen layout information storage buffer 10 and output voice and Braille. A data sequence is generated, the data sequence is converted into voice and Braille, and voice and Braille are output.

4) In the screen layout information analysis means 13, the movement of the point of interest such as the mouse cursor is detected, and the moving direction of the point of interest and the distance from the window, icon or the like in that direction are detected. Seeking from information,
A buzzer sound, which is an intermittent tone, is output according to the distance.
In this way, by controlling the voice, even visually impaired people can
The distance between the point of interest and the icon can be identified by the pitch and interval of the sound.

5) When there is a request from the user to output windows, icons, etc. around the point of interest such as the mouse cursor (detection of pressing of a predetermined control key A on the output operation section 6), the point of interest The screen layout information storage buffer for the screen layout information at the position of or around 1
Import from 0, convert the direction, distance and name of each window and icon into voice and Braille data and output voice and Braille. At this time, the order of the output (for the point of interest,
The clockwise order) may be freely set.

6) Output request of screen position information from user (detection of pressing of predetermined control key B on output operation unit 6)
If there is, the screen layout information of the point of interest is fetched from the screen layout information storage buffer 10,
Convert the absolute position of the window or icon at the point of interest into voice or Braille data so that voice or Braille can be output.

7) In the screen layout information analysis means 13, changes in the shape of the point of interest such as the mouse cursor (for example,
Depending on where the mouse cursor is on the border of the window, the "up / down" cursor when it is on the horizontal border, the "left / right" cursor when it is on the vertical border,
When it comes to the boundary line of the corner, a "diagonal" cursor is displayed.) So that the state can be announced. Alternatively, the notification may be made by an instruction such as a key operation.

8) When the target window, icon name, coordinates, etc. are output in voice or Braille, and when the operator clicks the mouse or the like to indicate that the target icon or window has been found, the point of interest is noted. Move (skip) the cursor showing to the desired icon or window.

As a method for finding the above-mentioned desired icon and window, a key input is performed while reading the desired icon and window name, a number is given in the reading order, and the desired icon and window number are input. Etc. are possible.

In this way, by controlling the window on the multi-window, the name of the icon, the coordinates, the target window, the distance to the icon, etc., to be output to the operator by voice or Braille to notify the operator. Even a visually impaired person can use the information processing apparatus operating in the multi-window system, and in the information processing work of the visually impaired person, it greatly contributes to improvement in efficiency. Further, it is also effective for general sighted persons and elderly persons with reduced visual acuity in that the screen information can be confirmed by voice or the like.

[0040]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described in detail below with reference to the drawings. FIG. 1 described above is a principle configuration diagram of the information processing apparatus of the present invention, FIGS. 2 to 5 are diagrams showing an embodiment of the present invention, and FIG. 2B shows an example of a format of history information of cursor coordinate data, and FIG.
Shows a flow chart of the processing in the screen layout information analysis means 13, FIG. 3 shows the main processing, and FIG.
An example of processing corresponding to a control key detected by the output operation unit is shown, and FIG. 5 shows an example of windows on the display screen, display examples of icons, and examples of mouse cursor shapes.

In the information processing apparatus of the present invention, the system, application 2 requests the OS unit 3 to move the mouse cursor, output windows, icons, etc.
The screen layout information you are trying to output by hooking at the entrance to 3 (as shown in Figure 2,
(Position coordinates of the window and icon, area data for displaying the window and icon, image data for displaying in the area, character code, cursor code) are loaded and saved in the screen layout information save buffer 10. Means for moving the mouse cursor and for outputting the screen layout information above the point of interest such as the mouse cursor from the screen layout information storage buffer 10 when there is an output request from the user (screen layout information reading means) 8. Screen layout information analysis means 13 for analyzing the screen layout information extracted from the screen layout information storage buffer 10 to generate a data string for outputting a predetermined voice or Braille, the screen layout information analysis means 13 The movement of the point of interest such as the mouse cursor is detected in the By storing the history of coordinates, the moving direction of the mouse cursor can be calculated), and the name, direction, distance, etc. of the window, icon, etc. in the moving direction of the point of interest can be displayed on the above screen. The contents of the layout information storage buffer 10 are analyzed to generate a data string necessary for outputting voice and Braille, and a means for converting the data string into voice and Braille to output voice and Braille is used. It is a necessary means for carrying out the invention. Note that the same reference numerals indicate the same object throughout the drawings.

The configuration and operation of the information processing apparatus of the present invention will be described below with reference to FIGS. 2 to 5 with reference to FIG. As shown in FIG. 1, the information processing apparatus according to the present invention includes a screen creation request from the system and application 2 when the information processing apparatus is activated, a mouse cursor of a user's mouse and keyboard operation. A window corresponding to movement, an operating system (OS) unit 3 that controls the screen in response to an icon output request, a display 5 that outputs a screen corresponding to the screen control, a voice / Braille output unit 7, and an output operation unit 6 and a speaker / voice synthesizer / Braille display device 12
Braille output means 7 hooks the mouse cursor movement information, window and icon output information sent from the application 2 to the operating system (OS) 3.
(Preemption) and save the screen layout information buffer
Corresponding to the screen layout information storage means 9 to be stored in 10 and the control key information from the output operation unit 6 which detects that a predetermined control key on the keyboard 1 has been pressed, the screen layout information of the predetermined area is stored. , The screen layout information reading means 8 which is read from the screen layout information storing means 9 and is passed to the screen layout information analyzing means 13 and the data string output from the screen layout information analyzing means 13 are converted into voice and Braille data strings. And output it to the speaker / speech synthesizer / Braille display device 12 for voice, Braille, on-screen windows, icon names,
It is composed of a voice / Braille data conversion / buzzer control unit 11 for notifying the user of coordinates, distance and the like.

The screen layout information is the information of the format shown in FIG. 2A, and the coordinates (for example, the upper left corner coordinates) of the window and the icon, the area data, and the window and the icon are visually recognized in the area. Image data to be displayed dynamically, and character codes to display the names of the windows and icons.

Also, as shown in FIG. 2B, the cursor information has the same format, for example, past cursor information and current (current) cursor information, and the coordinate data thereof is It changes momentarily by the user's mouse operation. Further, when the central processing unit (CPU) (not shown) of the information processing device is performing a predetermined process and the mouse input is not possible, the application 2
For example, change it to a cursor that looks like a clock,
As described above, when the mouse cursor moves on the boundary line of the window, it is changed to a cursor having a predetermined shape depending on whether it is on the horizontal boundary line, the vertical boundary line, or the corner boundary line. However, the cursor information is output as mouse cursor information from the corresponding application 2 and the screen output means 4 of the operating system (OS) 3 analyzes the cursor information to display the shape of the corresponding cursor. Display in overwrite format on the screen of 5.

In the information processing apparatus of the present invention, the above screen layout information is transmitted from the corresponding application 2
When it is sent to the operating system (OS) 3, the screen layout information storage means 9 hooks (takes in) and stores it in the screen layout information storage buffer 10.

The screen layout information reading means 8 reads, for example, the entire screen layout described above each time the screen layout information storage buffer 10 is updated, or the output operation unit 6 to be described later controls a predetermined operation by the user. Each time a key press is detected, the screen layout information in a predetermined area is read and sent to the screen layout information analysis unit 13.

The operation of the screen layout information analyzing means 13 will be described below with reference to FIGS. First, when the information processing apparatus is powered on and a predetermined program is initially loaded and the information processing apparatus starts up, a corresponding system program (for example, a screen generation program) 2 operates, and a window, icon, mouse cursor Output request and corresponding screen layout information are output to the operating system (OS) section 3, and the screen output means 4 displays the screen on the screen of the display 5, for example, as shown in FIG.
Display the screen as shown in (a). In FIG. 6 (a), a frame displayed with a double line indicates a window (for example, program manager, main), and a graphic in each window indicates an icon (pictogram).
The characters displayed under (Icon) indicate the name of each icon, and display the graphic data created from the screen layout information described in FIG.
The screen is displayed by drawing in a bitmap memory (not shown) in 5. The cursor display is also displayed by the same process.

In the information processing apparatus of the present invention, the above-mentioned screen layout information sent from the application 2 to the operating system (OS) 3 is output as voice / Braille output means.
The screen layout information storage means 9 in 7 is hooked (preempted) and stored in the screen layout information storage buffer 10, but new screen layout information is hooked and stored in the screen layout information storage buffer 10.
Each time (updated), screen layout information reading means
8 reads the contents of the screen layout information storage buffer 10 and transfers it to the screen layout information analysis means 13.

When the user operates the mouse cursor while the predetermined application program 2 is being executed,
The movement of the mouse cursor occurs, the screen layout information of the cursor is sent to the operating system (OS) 3 together with the cursor output request, and the screen layout information is sent to the screen layout information analysis means 13.

3 and 4 show an example of the processing flow in the screen layout information analysis means 13, and FIG. 3 shows the flow of the main processing. First, the system program (the above-mentioned screen creation program), application 2
Generates a data string for outputting the window, icon name, position, and cursor position of each window, icon, and cursor from the screen layout information requested by the user, from the current window, icon, and mouse cursor screen layout information in voice or Braille. Then, it is sent to the voice / Braille data conversion / buzzer control unit 11 and converted into voice / Braille data by the voice / Braille data conversion / buzzer control unit 11 and output as voice / Braille.

At this time, the screen layout information of the current mouse cursor is compared with the character information of the past screen layout information, and the character information is, for example, a normal "arrow".
Changed from the cursor to the cursor code that indicates a "clock-like shape" that indicates that the central processing unit (CPU) of the information processing device is performing a predetermined process and is unable to accept mouse input When it is detected, the shape of the mouse cursor is output as voice or Braille to notify the user that "mouse input is currently impossible". {Refer to processing step 100 in FIG. 3} Next, when the cursor movement occurs in response to the user's mouse operation and a cursor output request is output, it is checked whether the current cursor is on the window or icon. The coordinates of the screen layout information of the cursor are compared with the screen layout information of the window and icon (see processing step 102 in FIG. 3), and the corresponding screen layout information is detected on the window and icon (coincidence of coordinate detection). Is extracted, the window pointed by the current cursor, the name of the icon, the coordinates, etc. are converted into voice and Braille data, and voice and Braille are output.

At this time, whether or not the current cursor is on the boundary line of the window is judged from the coordinates of the screen layout information of the cursor shown in FIG. 2 (b). It is possible to discriminate whether it is on the border line of, the vertical border line, or the corner border line (this discrimination process can recognize the border line by the coordinate information of the window and the area information, and It can be determined by comparing with the coordinate information of the mouse cursor), recognizes where the mouse cursor is on the boundary line of the window, and the shape of the predetermined cursor (up, down, left, right, diagonal) is spoken or Braille. By outputting, the user can recognize where the current mouse cursor is on the border line on the window. {Refer to the processing steps 101, 102, 103 of FIG. 3} In the judgment processing of the above processing step 102, when the current cursor is not on the window or icon, the movement amount of the mouse cursor is changed to that of the mouse cursor shown in FIG. 2 (b). When the amount of movement is small, calculated from the past coordinates and the current coordinates, the window and icon around the cursor are displayed as shown in FIG.
Extracted from the screen layout information of (a), its name,
Coordinates, directions (eg, up / down, left / right, diagonal, etc.), distances (eg, movement distance of mouse 1), etc. are converted into voice and Braille data, and voice and Braille are output. At this time, the output order may be specified such that the mouse cursor is rotated clockwise or the like. {Refer to processing steps 104 and 105 in FIG. 3} When it is determined in the cursor movement amount calculation processing (see processing step 104) that the movement amount is large, the movement direction of the mouse cursor is set to the cursor in FIG. Calculated from the coordinates of the screen layout information of, the windows and icons in the moving direction are extracted from the screen layout information of FIG. (For example, moving distance of mouse 1) is converted into voice and Braille data, and voice and Braille are output.
{Refer to processing steps 104, 106, 107 in FIG. 3} At this time,
The distance may be output with a pitch or an intermittent buzzer sound according to the calculated distance. In this case, from the current mouse cursor to the window and icon at the pitch height and interval. The distance of can be recognized.

The above processing is repeated each time the mouse cursor is moved. In this case, only when the movement of the mouse cursor is detected, the voice and the Braille are output once, and the next operation is performed. Until the mouse cursor is moved, the user cannot recognize the current position of the mouse cursor.

Therefore, in the information processing apparatus of the present invention, FIG.
As shown in, the screen layout information of the predetermined area is read from the screen layout information storage buffer 10 on the basis of the information (output request) from the output operation unit 6, and the screen layout information analysis means 13 reads the screen layout information. I am sending it out.

FIG. 4 shows a processing flow at this time. When the output operation unit 6 detects that the user presses a predetermined control key A (first control key) on the keyboard, , The processing steps of the main processing described in FIG.
The same process as 103 and 105, that is, the windows and icon names on and around the mouse cursor,
Coordinates, distances, etc. are output as voice and Braille. With such a configuration, the user only has to press the predetermined control key A (first control key), and the window on and around the mouse cursor, the names of icons,
Coordinates, distances, etc. can be recognized. {Refer to processing steps 200 and 201 in FIG. 4} When the output operation unit 6 detects that the user presses a predetermined control key B (second control key) on the keyboard, the mouse cursor is present at present. The coordinates of the current position and the names and coordinates of the window, icon, etc. at that position are output in voice or Braille. With such a configuration, the user can recognize the current position of the mouse cursor and the names of the window and icon at that position by pressing the predetermined control key B many times. {Refer to the processing steps 300 and 301 of FIG. 4} As described above, the voice, the Braille output window,
When the name of the icon is recognized and the target window or icon is found, a predetermined control key C on the user's keyboard is pressed or the mouse 1 is clicked to select it.

As described above, the selecting operation is performed by pressing the predetermined control key C when the name of the target window and icon is output in voice or Braille, or by assigning numbers in the reading order. There are means for inputting numbers corresponding to target windows and icons.

When the output operation section 6 detects a predetermined control key C (third control key) on the keyboard or a mouse click operation by the user, the current mouse cursor is moved to the control key C Skip (move) to the position of the window or icon selected by operation etc. (this position can be recognized by looking at the coordinates of the screen layout information shown in FIG. 2 (a) above). To control. In this way, the target window and icon can be easily selected. {Refer to processing step 400 of FIG. 4} When the output operation unit 6 detects that the user presses a predetermined control key D (fourth control key) on the keyboard, the current shape of the mouse cursor is voiced. , Output in Braille. With such a configuration, the user can confirm the current shape of the mouse cursor by pressing the predetermined control key D many times. {Refer to processing step 500 of FIG. 4} FIG. 5 shows a display example of the screen of the display 5.
FIG. 5 (a) is a display example of the initial screen when the information processing apparatus starts to operate, and the windows "Program Manager" and "Main" and the menus in them are the icons "File Manager" and "File Manager". Control panel"
It is displayed by etc.

In the case of the display example of FIG. 5 (a), when the cursor (arrow) at the point of interest moves to the "control panel" icon, "control panel nowedes" is output. According to the means described in 3) of the "means" column, the window or icon name in the moving direction of the current point of interest is output as voice or Braille. If there are a plurality of icons in the moving direction, a plurality of voices and Braille may be output in order from the one with the shortest distance. When there is no icon in the moving direction, voice or Braille may be output indicating that there is no icon.

In the case of the display example of FIG. 5 (a), when the cursor at the point of interest is moved to the right, "Control panel print manager clipboard gallery" is output. At this time, as the distance between the target and the cursor at the point of interest is shorter, for example, a higher buzzer sound is output. Conversely, as the distance is longer, for example, a higher buzzer sound may be output.

Further, the shorter the distance between the target and the point of interest due to the continuous buzzer sound, the smaller the interval, and the longer the distance, the longer the interval may be. According to the means described in 5) of the "means" column, the user's specific control key A (first control key)
You can output the window or icon name at a certain distance around the point of interest by voice or Braille by the operation. To explain the case of the display example of FIG. 5 (a), by a specific control key operation, for example, with the displayed mouse cursor as the center, clockwise, "Control Panel Windows Setup MS Dos Prompt File Manager "Arimas" is output.

If the windows and icon names within a certain distance around the point of interest are output by voice and Braille in the order closer to the point of interest by the user's key operation, the direction and name of the window and icon are output in Braille. In the case of the display example of (a), by operating the control key, "Migi Control Panel
File Manager Mishitita Windows Setup Hidarishita MESDOS Prompt Gaarimas ”is output.

According to the means described in 6) of the "means" column, the coordinate value of the position where the mouse cursor of the target point is located by the operation of the specific control key B (second control key) by the user, Coordinate values of the window and icon at the position of the mouse cursor are output as voice and Braille. Coordinates are X coordinate, Y
The dot value of coordinates and the actual operation distance of mouse 1 are output in order. In the case of the display example of FIG. 5A, "control panel 100 dots 150 dots death" is output.

According to the means described in 7) of the "means" column, when there is a change in the shape of the point of interest such as the mouse cursor, that state, for example, the mouse cursor shown in FIG. Announce the shape "up and down", "oblique" and "left and right". Further, the state may be notified by an instruction such as an operation of the specific control key D (fourth control key).

According to the means described in 8) of the "means" column, the point of interest can be moved to the target icon or window by key operation. In the case of the display example described in FIG. 5 (a), a specific control key C (third control key)
If you perform the key operation while reading "Clipboard" by reading or the mouse click operation, the mouse cursor at the point of interest moves (jumps) onto the "Clipboard" and the icon "Clipboard" Can be selected.

As described above, the information processing apparatus of the present invention outputs information such as icons in the multi-window system to the voice,
An information processing device having means for outputting in braille, in which a mouse cursor is moved from an application program,
Based on the means for hooking (capture) the screen layout information associated with the window and icon output request, and saving it in the screen layout information save buffer, and the output request from the mouse cursor or the mouse or keyboard 1. A means for extracting predetermined screen layout information from the layout information storage buffer, analyzing the extracted screen layout information, and surrounding the point of interest,
Alternatively, a means for outputting a window in the moving direction of the point of interest, an icon name, coordinates, a distance, etc., and a means for outputting a data string of braille and a means for converting the data string of the voice, braille to a sound, and outputting the braille. It is characterized in that it is provided with voice and Braille output means.

[0066]

As described above, according to the information processing apparatus of the present invention, the visually impaired person can use the information processing apparatus operating in the multi-window system.
In the information processing work of the visually impaired, it greatly contributes to the improvement of efficiency.

Further, the target user is not limited to the above-mentioned visually impaired person, and the screen information can be confirmed by voice or the like.
It is also effective for general sighted people and elderly people with reduced visual acuity.

[Brief description of drawings]

FIG. 1 is a principle configuration diagram of an information processing apparatus of the present invention.

FIG. 2 shows an embodiment of the present invention (part 1).

FIG. 3 shows an embodiment of the present invention (part 2).

FIG. 4 shows an embodiment of the present invention (part 3).

FIG. 5 shows an embodiment of the present invention (part 4).

FIG. 6 illustrates a conventional information processing device.

[Explanation of symbols]

1 mouse, keyboard 2 system, application, system program, application program 3 operating system (OS) section 4 screen output means 5 display 6 output operation section 7 voice / braille output means 8 screen layout information reading means 9 screen layout information storage means 10 Buffer for saving screen layout information 11 Voice / Braille data conversion / buzzer control unit 12 Speaker / Speech synthesizer / Braille display device 13 Screen layout information analysis means Hook (preemption) Screen layout information Information data such as windows and icons Data string from keyboard Output request Means to move the mouse cursor to the selected window or icon 100 to 107,200 to 201,300 to 301,400,500 Processing steps

Claims (9)

[Claims] 1. An information processing apparatus having means for outputting information such as icons in a multi-window system in voice or Braille, which is output in response to a mouse cursor movement from an application program, a window or icon output request. Means for hooking screen layout information to be stored in the screen layout saving buffer, and means for extracting predetermined screen layout information from the screen layout saving buffer based on a movement of a mouse cursor, an output request from a mouse or a keyboard, A means for analyzing the retrieved screen layout information and outputting a data string for instructing information such as windows and icons on the screen, and a voice and braille output means for converting the data sequence into voice and Braille for output. An information processing apparatus comprising: 2. The information processing apparatus according to claim 1, wherein
The information processing apparatus, wherein the information such as windows and icons on the screen is names and coordinates of windows and icons on or around a point of interest pointed by a mouse cursor. 3. The information processing apparatus according to claim 1, wherein
The detecting means for detecting the moving direction when the point of interest pointed by the mouse cursor moves to the target is provided, and the information such as the window on the screen and the icon is the window in the direction detected by the moving direction detecting means. , An information processing device characterized by the name, direction, distance, etc. of an icon or the like. 4. The information processing apparatus according to claim 3, wherein
An information processing apparatus comprising: means for changing the scale of a buzzer sound and outputting a sound corresponding to a distance between a target point and a target pointed by the mouse cursor. 5. The information processing apparatus according to claim 1, wherein
A means for detecting a predetermined first key operation is provided, and based on an instruction by the predetermined first key operation detected by the first key operation detecting means, the periphery of the point of interest pointed by the mouse cursor An information processing device, which outputs the name, coordinates, and distance of a window, an icon, and voice in Braille. 6. The information processing apparatus according to claim 1, wherein
Means for detecting a predetermined second key operation is provided, and the position of the point of interest pointed by the mouse cursor is based on the instruction by the predetermined second key operation detected by the second key operation detecting means. An information processing device, which outputs the coordinates of, and the names and coordinates of windows and icons above the point of interest in voice and Braille. 7. The information processing apparatus according to claim 1, wherein
An information processing apparatus, wherein when the analysis unit detects the screen layout information for a change in the shape of a mouse cursor displayed by the information processing apparatus, the change in the shape of the mouse cursor is output as voice or Braille. 8. The information processing apparatus according to claim 1, wherein
A means for detecting a predetermined third key operation is provided, and based on the predetermined third key operation detected by the third key operation detecting means, the shape of the current mouse cursor displayed by the information processing device. An information processing apparatus, which outputs a voice or Braille. 9. The information processing apparatus according to claim 5, wherein
When a means for detecting a predetermined fourth key operation is provided and the window or icon output in voice or Braille is selected by the key operation detected by the fourth key operation detecting means, the mouse of the attention point An information processing apparatus comprising means for moving a cursor onto the selected window or icon.