KR20020058155A - Control method of computer program used voice recognition technology - Google Patents

Abstract

PURPOSE: A method for reading of computer screen information and controlling a voice of program using voice synthesis and voice recognition is provided to enable a handicapped person to use a computer easily by reading screen information being displayed on a computer screen as a voice and controlling an operating system and programs of the computer by a voice only. CONSTITUTION: A computer is booted(S100). An OS program is started(S110). A message hook is set(S120). A message hooking is started(S130). It is checked whether voice is inputted(S140). If the voice is inputted, screen information being displayed on a screen is read as a voice(S200). If the voice is inputted, a program is controlled as a voice(S300). If the voice is not inputted, the current stage is returned to the above stage(S200,S150).

Description

Speech reading of computer screen information and speech control method using speech synthesis and speech recognition {Control method of computer program used voice recognition technology}

The present invention relates to a voice reading method of a computer screen information and a voice control method of a program using voice synthesis and voice recognition, and more particularly, to read the screen information displayed on a computer screen with a voice and The present invention relates to a voice reading method of a computer screen information and a voice control method of a program using voice synthesis and voice recognition, in which not only a normal person but also a visually impaired person can use a computer very easily by controlling only a voice.

In general, computers are indispensable in living an information society, and how proficient and effective use of such computers is regarded as the ability of modern society to live in an information society.

Therefore, in recent years, due to the development of OS programs such as Windows, which is not only highly trained to learn a computer, but also the development of an OS (operation system) program that connects a computer and a user, the computer is easier to learn and very simple to use. Computer population is growing exponentially

However, despite the rapid development of the above computer and OS programs, the visually impaired as well as the various handicapped people who are unable to use their hands or sit on their desks, despite the fact that they are not able to use or use computers, Nevertheless, the use of computers within a very limited range has led to a wider gap in ability with normal people, which forced them to be rapidly removed from society. Many efforts have been made to overcome these problems. As a result of the present application, a method for enabling a visually impaired person to use a computer simply and easily by outputting various screen information of a computer operated by an OS program such as Windows according to a user's setting is "a visually impaired person. for It was filed with the Korean Patent Office on June 28, 2000 under the name of "Computer Screen Reading Method" and was granted the application number of Patent Application No. 2000-35966 (hereinafter referred to as "first application"). The patent application was filed with the Korean Intellectual Property Office on October 16, 2000 under the name of "Control method of computer program used" and the application number of patent application No. 2000-60732 (hereinafter referred to as "second application").

The first application is to visually display the object information on the screen to a visually impaired person who cannot see the screen of the monitor so that the system information and various program information can be easily known. The second application is a hand such as a mouse and a keyboard. It is possible to control the operating system and various programs of the computer by voice commands without using the input means.

However, the above-described applicants can make it possible to use the computer very easily for the disabled as well as the normal person, but since the two programs must be executed separately, there is a problem in that the disabled person is uncomfortable using the computer freely.

The present invention has been made in view of the problems of the above-described application, the purpose of which is to read the screen information displayed on the computer screen by voice and at the same time by controlling the operating system and programs of the computer by voice only, as well as the time and various It is to provide a method of voice reading of computer screen information and voice control of a program using voice synthesis and voice recognition, which enables the disabled to use a computer very easily.

An object of the present invention is to read a computer screen information using speech synthesis and speech recognition and a voice control method of a program, the method comprising the steps of booting a computer; Starting an OS program; Establishing a message hook; Initiating message hooking; Checking whether there is a voice input; A computer screen reading step of reading the voice out screen information displayed on the screen when it is confirmed that there is no voice input in the checking step; A voice control step of controlling a program by voice when it is confirmed that there is a voice input in the checking step; And confirming whether there is a voice input, and if there is a voice input, returning to the voice control step, and if there is no voice input, the voice input reconfirming step of returning to the computer screen reading step. It can be achieved by the voice reading of the computer screen information using the voice recognition and the voice control method of the program.

1 is a flowchart illustrating a method of reading a voice of computer screen information and controlling a voice of a program using voice synthesis and voice recognition according to the present invention.

2 is a flowchart illustrating a computer screen reading step of a voice reading method of a computer screen information and a voice control method of a program using voice synthesis and speech recognition according to the present invention.

Figure 3 is a flow chart illustrating another embodiment of the computer screen reading step of the voice reading method of the computer screen information and the program using the speech synthesis and speech recognition in accordance with the present invention.

4 is a flowchart illustrating a menu content voice output step of a computer screen step of a voice reading method of a computer screen information and a voice control method of a program using voice synthesis and voice recognition according to the present invention.

5 is a flowchart illustrating a text content reading step of a computer screen reading step of a voice reading method of a computer screen information and a voice control method of a program using voice synthesis and speech recognition according to the present invention.

6 is a flowchart illustrating a voice control step of a voice reading method of a computer screen information and a voice control method of a program using voice synthesis and voice recognition according to the present invention.

7 is a flowchart illustrating a text voice input step of the voice control step of the voice reading method of the computer screen information and the voice control method of the program using voice synthesis and voice recognition according to the present invention.

Hereinafter, a voice reading method of computer screen information and a voice control method of a program using voice synthesis and voice recognition according to a preferred embodiment of the present invention will be described with reference to the accompanying drawings.

1 is a flowchart illustrating a method of reading a computer screen information using voice synthesis and voice recognition and a voice control method of a program according to the present invention, and FIG. 2 is a view illustrating the computer screen information using voice synthesis and voice recognition according to the present invention. 3 is a flowchart illustrating a computer screen reading step of a voice reading method and a voice control method of a program, and FIG. 3 is a computer screen reading step of a voice reading method of a computer screen information and a voice control method of a program using voice synthesis and voice recognition according to the present invention. Figure 4 is a flow chart illustrating another embodiment of Figure 4 is a flow chart illustrating the menu content voice output step of the computer screen step of the voice reading method of the computer screen information and the voice control method of the program using the speech synthesis and speech recognition in accordance with the present invention 5 is a speech reading of computer screen information and speech of a program using speech synthesis and speech recognition according to the present invention. FIG. 6 is a flowchart illustrating a text content reading step of a computer screen reading step of the method, and FIG. 6 illustrates a voice control step of a voice reading method of a computer screen information and a voice control method of a program using voice synthesis and voice recognition according to the present invention. 7 is a flowchart illustrating a text speech input step of a voice control step of a voice reading method of a computer screen information and a voice control method of a program using voice synthesis and voice recognition according to the present invention.

1 to 7, the voice reading method of the computer screen information and the voice control method of the program using the speech synthesis and speech recognition according to the present invention

Computer booting step (S100), OS program start step (S110), message hook setting step (S120), message hooking step (S130), voice input confirmation step (S140), computer screen reading step (S200) And a voice control step (S300) and a voice input re-check step (S150).

The computing step (S100) maintains the computer system in a usable state by turning on the power of the computer, and the OS program starting step (S110) is an OS program, that is, an operation for allowing a user to easily control the computer. Run the program.

The message hook setting step (S120) executes a program to set a message hook, and the message hooking step (S130) starts the message hooking when the message hook setting step (S110) is completed.

The voice input confirmation step (S140) checks whether a voice command is input, and if the voice command is not input, performs the computer screen reading step (S200), and if the voice command is input, the voice control step (S300). Is carried out.

The computer screen reading step (S200) includes an object information reading step (S210), an object information storing step (S220), an object information transmitting step (S230), and an audio output step (S240).

The object information reading step S210 reads the object information on the screen, and the object information storing step S220 stores the object information read in the object information reading step S210 in a buffer.

The object information transmission step (S230) transmits the stored object information to the voice engine.

The voice output step (S240) outputs the object information transmitted to the voice engine as voice through the output means.

The computer screen reading step S200 includes an event occurrence checking step S250, a cursor coordinate information reading step S260, and an object checking step S270.

The event occurrence checking step (S250) checks whether an event in which the cursor or the mouse pointer moves on the screen is generated by an input means such as a mouse or a keyboard, and if the event is not generated, immediately reading the object information (S210). ) To read the entire object information on the screen.

The cursor coordinate information reading step (S260) reads the coordinate value where the cursor is located when an event occurs in the checking step (S250). When the coordinate information of the cursor is read as described above, the object checking step (S270) checks whether an object exists on the read cursor coordinate, and if the object does not exist, returns to the event occurrence checking step. If an object exists on the cursor coordinate in the object checking step (S270), information on the object located on the cursor coordinate is read in the object information reading step (S210).

In addition, the computer screen reading step (S200) is a menu content audio output step (S400) that can output a voice by distinguishing a menu item having a submenu and a menu item without a submenu among menu items of an OS program or various programs. Include.

The menu contents voice output step S400 includes a male voice setting check step S410, a female voice set check step S420, an event generation step S430, a submenu check step S440, and a basic voice output step. Step S450 and sub-audio output step S460.

The male voice setting confirmation step (S410) is to determine whether to set the default voice, that is, the voice when outputting the contents of the item without the sub-menu to the male voice, if the user selects the female voice setting confirmation step It is moved to S420. If the user sets the basic voice to the male voice, when the user outputs the sub voice, that is, the menu contents with the sub menu, the voice is output as the female voice.

The female voice setting confirmation step (S410) is to determine whether to set the default voice, that is, the voice when the output of the contents of the item without the sub-menu to the female voice, if the user selects the male voice setting confirmation step It is moved to S410. If the user sets the basic voice to the male voice, the voice is output as the sub voice, that is, when the menu contents with the sub menu are voice output.

The event generating step (S430) is to read the contents by moving the reading cursor using an input device such as a keyboard or mouse when the default voice is set by selecting YES in any one of the checking step (S410) (S420) Raises an event. Move the cursor or mouse pointer to position on any menu item.

The sub-menu confirmation step (S440) checks whether there is a submenu in the designated menu item when the event generation step (S430) is executed as described above, and if the menu item has no submenu, the basic voice output step (S450). If the sub menu has a menu item, the sub voice output step (S460) is performed.

In the basic voice output step (S450), when there is no submenu in the submenu checking step (S440), the contents of the menu item corresponding to the voice set as the basic voice in the voice setting checking step (S410) (S420) are spoken. Output That is, when the basic voice is set to male voice, the menu contents are output as male voice, and when the basic voice is set to female voice, the menu contents are output as female voice.

The sub-audio output step (S460) is a voice of the contents of the menu item as a sub-voice not set as the basic voice in the voice setting confirmation step (S410) (S420) when there is a sub-menu in the sub-menu confirmation step (S440). Output In other words, when the basic sound is set to male voice, the contents of the menu item are output as voice by female voice. When the basic voice is set as female voice, the contents of menu item is output by voice.

In addition, the read computer screen reading step (S200) includes a text content reading step (S500) so that the text content can be output voice.

The text content reading step (S500) includes a reading unit setting step (S510), an event generating step (S520), a content reading step (S530), a reading content storing step (S540), a content transmitting step (S550), , Audio output step (S560).

In the reading unit setting step (S510), the user sets the reading unit of the text content. That is, the unit sets the unit in which the user outputs the text such as one note, one letter, one word, one syllable, one line, and one phrase.

The event generation step (S520) generates a text read event, and moves the cursor to a position to be read by moving the cursor.

The content reading step S530 reads the text content in order to output the text content by voice.

The reading contents storing step S540 stores the contents read in the contents reading step S530 in a buffer.

The content transmission step (S550) transmits the stored content to the voice engine, and the voice output step (S560) outputs the content transmitted to the voice engine through voice output means.

The voice control step (S300) is to control a program by voice when it is confirmed that there is a voice input in the checking step (S140),

Voice command recognition confirmation step (S310), re-input message output step (S320), voice command comparison step (S330), voice command registration confirmation step (S340), and the command execution step (S350). .

The voice command recognition confirmation step (S310) checks whether the voice command input correctly when the voice input is confirmed in the voice input confirmation multiplication system (S140), and outputs the re-input message when the voice command is not recognized. Step S320 is performed.

The re-input message output step (S320) outputs a message requesting the user to input the voice command again when the voice command is not recognized in the checking step (S310), and returns to the voice input checking step (S140). do.

The voice command comparing step S330 compares whether or not the voice command is identical to the voice command already registered in the data unit when the voice command input in the checking step S310 is recognized.

The voice command registration check step (S340) checks whether the voice command registered in the voice command comparison step (S330) is the same as the input voice command, and if there is no identical input message output step (S320). If the same voice command is returned, the corresponding command execution step (S350) is executed to execute a command corresponding to the registered voice command.

In addition, the voice control step S300 includes a text voice input step S600 for allowing text to be input by voice without using a keyboard on a text window or a document preparation field of a text program.

The text voice input step (S600) is a voice input step (S610), voice command recognition confirmation step (S620), voice re-input request step (S630), voice command comparison step (S640), voice command registration confirmation A step S650, a text program execution checking step S660, a program execution request step S670, a cursor position checking step S680, a grammar verification step S690, a text combination step S690a, It consists of a text output step (S690b).

The voice input step (S610) voice inputs a keyboard control command in order to voice input text into a text window or a text program.

The voice command recognition check step S620 confirms whether the input voice command is correctly recognized, and if it is not recognized, the voice re-input request step S630 is executed.

The voice re-input request step (S630) outputs a re-input message when the voice command input in the check step (S620) is not recognized and returns to the voice input step (S610).

The voice command comparing step S640 compares the voice command registered in the voice data unit with the input voice command when the voice command input in the checking step S620 is recognized.

The voice command registration confirmation step (S650) checks whether the voice command word inputted to the voice data unit in the comparison step (S640) is registered, and if not registered, outputs a message indicating that the command is not registered. Return to the voice re-input request step (S640).

The text program execution confirmation step (S660) confirms whether the text program is being executed when the registration of the voice command is confirmed in the confirmation step (S650).

The program execution request step S670 outputs a message indicating that the program is not being executed when the program is not being executed in the confirmation step S660, and returns to the voice input step S610.

The cursor checking step (S680) checks the cursor position on the screen when the program is running in the checking step (S660).

The grammar verification step S690 verifies the input voice command by grammar dictionary.

The text combining step S690a combines the text corresponding to the voice command according to the result of the grammar verification step S690.

The text output step S690b outputs the text in which the combination is completed in the text combining step S690a on a text field or a text composability field.

The voice input re-check step (S150) checks whether a voice command is input during the execution or the end of the computer screen reading step (S200), and if the voice command is not input, continues the computer screen reading step (S200). If the voice command is input, the computer screen reading step S200 is stopped, and the voice control step S300 is immediately performed.

In addition, when the operating system or a program is controlled by a voice command or a text is input or a text document is created in the voice control step (S300), all object information or text content is executed by the computer screen reading step (S200). Can be output as

According to the present invention described above, by reading the screen information displayed on the computer screen with a voice, and controlling the operating system and programs of the computer by voice only, it is possible to use the computer very easily even for the normal and visually impaired. Has

Claims (8)

A voice reading method of a computer screen information and a voice control method of a program using a combination of voice synthesis and voice recognition, the method comprising: booting a computer (S100); Starting an OS program (S110); Setting a message hook (S120); Initiating message hooking (130); Checking whether there is a voice input (S140); A computer screen reading step (S200) of reading out voice of screen information displayed on a screen when it is confirmed that there is no voice input in the checking step (S140); A voice control step (S300) of controlling a program by voice when it is confirmed that there is a voice input in the checking step (S140); Check if there is a voice input, and if there is a voice input, returns to the voice control step (S300), and if there is no voice input, returns to the computer screen reading step (S200) to the voice input recheck step (S150). A voice reading method of a computer screen information and a voice control method of a program using voice synthesis and voice recognition, characterized in that the configuration. The computer screen reading step (S200) comprises: an object information reading step (S210) for reading object information on a screen; An object information storage step (S220) of storing the read object information in a buffer; An object information transmitting step (S230) of transmitting the stored object information to the voice engine; Speech reading and computer control method of the computer screen information using the speech synthesis and speech recognition characterized in that it comprises a voice output step (S240) for outputting the object information transmitted to the voice engine through the output means. The method of claim 2, wherein the computer screen reading step (S200) comprises: checking whether an event has occurred on the screen by an input means (S250); Reading cursor coordinate information when an event occurs in the checking step S250; Checking whether an object exists on the read cursor coordinates, and if an object does not exist, returning to the event occurrence checking step (S270); When the object exists on the cursor coordinate in the checking step (S270), the voice reading of the computer screen information using the speech synthesis and voice recognition, characterized in that the object information on the cursor coordinate is read in the object information reading step (S210) and Voice control method of the program. 4. The method of claim 3, wherein the input means is a mouse or a keyboard. The method of claim 2, wherein the computer screen reading step (S200) comprises: a male voice setting checking step (S410) for checking whether to set the basic voice as a male voice; Confirming whether to set the basic voice as the female voice when selecting NO in the checking step (S410), and if the NO is selected, checking the female voice setting step (S420) returned to the checking step (S410); An event generating step (S430) of generating a content reading event by moving a read cursor using an input device when YES is selected in any one of the checking step (S410) (S420); A submenu checking step (S440) of checking whether a menu on which the reading cursor is located has a submenu; A basic voice output step (S450) for outputting menu contents as voices set as basic voices in the voice setting checking step (S410) (S420) when there is no submenu in the submenu checking step (S440); When there is a sub-menu in the sub-menu confirmation step (S440), the voice setting confirmation step (S410) (S420) consists of a sub-audio output step (S460) for outputting the menu contents to the sub-voice not set as the default voice. A voice reading method of a computer screen information and voice control of a program using voice synthesis and voice recognition, characterized in that it comprises a menu content voice output step (S400). The method of claim 2, wherein the computer screen reading step (S200) comprises: a reading unit setting step (S510) for setting a reading unit of text contents; An event generation step (S520) of generating a content read event; A content reading step (S530) of reading the text content in order to output the text content as a voice; A read content storing step (S540) of storing the read content in a buffer; A content transmission step of transmitting the stored contents to the voice engine (S550); Voice of computer screen information using speech synthesis and speech recognition, characterized in that it comprises a text content reading step (S500) consisting of a voice output step (S560) for outputting the content transmitted to the voice engine through the output means. Reading and voice control of the program. The voice command recognition step (S310) of claim 1, wherein the voice control step (S300) comprises: a voice command recognition confirmation step (S310) for confirming whether a voice input is recognized; A re-input message output step (S320) outputting a re-input message of the voice command when the voice command input in the confirmation step (S310) is not recognized and returning to the voice input confirmation step (S140); A voice command comparison step (S330) of comparing the recognized voice command with the voice command registered in the data unit when the voice command is recognized in the checking step (S310); In the voice command comparison step (S330), it is checked whether there is the same voice command among the voice commands registered in step S330, and if there is no identical voice command, it is returned to the re-input message output step (S320). S340); Computer screen information using speech synthesis and speech recognition, characterized in that consisting of a command execution step (S350) for executing a command corresponding to the input voice command when the registration of the voice command is confirmed in the confirmation step (S340) Voice reading and voice control of the program. The method of claim 7, wherein the voice control step (S300) comprises: a voice input step (S610) for voice inputting a keyboard control command; A voice command recognition step (S620) of confirming whether the input voice command is recognized; A voice re-input request step (S630) for outputting a re-input message and returning to the voice input step (S610) when the voice command input in the confirmation step (S620) is not recognized; Comparing the voice command registered in the voice data unit with the input voice command when the voice command input in the checking step S620 is recognized (S640); In the comparison step (S640), it is checked whether the voice command word inputted to the voice data unit is registered, and if it is not registered, outputs a message indicating that the command is not registered and goes to the voice re-input request step (S640). A voice command registration confirmation step (S650) of returning; A text program execution checking step (S660) for checking whether a text program is being executed when the registration of the voice command is confirmed in the checking step (S650); A program execution request step (S670) for outputting a message indicating that the program is not being executed when the program is not being executed in the confirmation step (S660), and returning to the voice input step (S610); A cursor checking step (S680) of checking a cursor position on a screen when a program is executed in the checking step (S660); A grammar verification step (S690) of verifying an input voice command by a grammar dictionary; A text combining step (S690a) for combining text corresponding to a voice command according to the result of the grammar verification step (S690); The voice of the computer screen information using the speech synthesis and speech recognition, characterized in that it comprises a text speech input step (S600) comprising a text output step (S690b) for outputting the text of the combination is completed in the text combination step (S690a) Reading and voice control of the program.