JP2001075773A - Device and method for information processing and program storage medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To easily operate a program by recognizing a voice and generating a word corresponding to the voice, storing a command of a specific program corresponding to the word, and sending the command corresponding to the generated word to the program. SOLUTION: Data corresponding to a voice of a user are inputted and the data in specific form such as a text corresponding to a word that the user speaks are generated and supplied to a voice commander 102. A voice recognition engine 101 receives the recognized word or the data of grammar or the like from the voice commander 102 and stores them in a reading KANA (Japanese syllabary) dictionary database 111 or a recognized word and grammar database 112 for the engine. When the voice commander 102 is supplied with the data of the word or the like corresponding to specific speech that the user speaks from the voice recognition engine 101 sends a specific command to a still picture photography program 103, a still picture browsing program 104, or an electronic pet program 105.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

The present invention relates to an information processing apparatus and method, and a program storage medium, and more particularly to an information processing apparatus and method for recognizing voice, and a program storage medium.

[0002]

2. Description of the Related Art Information processing apparatuses such as personal computers operate various programs, and users can operate these programs.

[0003]

However, when operating a program operating on a personal computer or the like, it is necessary to operate an input device such as a keyboard or a mouse, which is troublesome.

[0004] The present invention has been made in view of such a situation, and an object of the present invention is to make it possible to easily operate a program.

[0005]

An information processing apparatus according to a first aspect of the present invention includes a generating means for recognizing a voice and generating a word corresponding to the voice, and a command of a predetermined program corresponding to the word. It is characterized by including a storage unit for storing and a transmission unit for transmitting a command corresponding to the word generated by the generation unit to the program.

[0006] The information processing apparatus may further include a selection unit for selecting a program from the running programs, and the transmission unit may transmit a command to the program selected by the selection unit.

According to a third aspect of the present invention, there is provided an information processing method comprising: a generating step of recognizing a voice and generating a word corresponding to the voice; a storing step of storing a command of a predetermined program in association with the word; A transmitting step of transmitting a command corresponding to the word generated in the processing of the generating step to the program.

According to a fourth aspect of the present invention, there is provided a program storage medium storing a program for recognizing a voice and generating a word corresponding to the voice, and storing a command of a predetermined program corresponding to the word. And a transmission step of transmitting a command corresponding to the word generated in the processing of the generation step to the program.

[0009] The information processing apparatus according to claim 1, claim 3,
In the information processing method described in the above, and in the program storage medium according to the claim 4, a word corresponding to the voice is generated by recognizing the voice, a command of a predetermined program is stored corresponding to the word, A command corresponding to the generated word is sent to the program.

[0010]

DESCRIPTION OF THE PREFERRED EMBODIMENTS One embodiment of an information processing apparatus according to the present invention will be described below with reference to the drawings.

FIGS. 1 to 6 show examples of the configuration of a portable personal computer to which the present invention is applied. The personal computer 1 is a mini-notebook type personal computer, and basically includes a main body 2 and a display unit 3 which can be opened and closed with respect to the main body 2. 1 is an external perspective view showing a state in which the display unit 3 is opened with respect to the main body 2, FIG. 2 is a plan view of FIG. 1, and FIG. 3 shows a state in which the display unit 3 is closed with respect to the main body 2. 4 is a right side view showing the display unit 3 opened 180 degrees with respect to the main body 2, FIG. 5 is a front view of FIG. 3, and FIG.
FIG.

On the main body 2, a keyboard 4 operated when inputting various characters and symbols, and a stick type pointing device 5 operated when moving a mouse cursor are provided on the upper surface thereof. Also,
On the upper surface of the main body 2, there are further provided a speaker 8 for outputting sound and a shutter button 10 operated when capturing an image with the CCD video camera 23 provided on the display unit 3.

A claw 13 is provided at the upper end of the display unit 3. As shown in FIG. 3, when the display unit 3 is closed with respect to the main body 2, Is provided with a hole 6 into which the claw 13 is fitted. A slide lever 7 is provided on the front surface of the main body 2 so as to be movable in parallel with the front surface. The slide lever 7 engages with a claw 13 fitted in the hole 6 to lock and unlock. I can do it. By releasing the lock, the display unit 3 can be rotated with respect to the main body 2. Microphone 2 next to claw 13
4 is attached. As shown in FIG. 6, the microphone 24 can collect sound from the back.

A programmable power key (PPK) 9 is also provided on the front of the main body 2. As shown in FIG. 4, an exhaust hole 11 is provided on the right side surface of the main body 2, and an intake hole 14 is provided at a lower part of the front surface of the main body 2 as shown in FIG. 5. Further, a PCMCIA (Personal Computer Memory Card Internat) is provided on the right side of the exhaust hole 11.
A slot 12 for inserting an ional association) card (PC card) is provided.

An LCD for displaying an image is provided on the front of the display unit 3.
(Liquid Crystal Display) 21 is provided, and an imaging unit 22 is provided at the upper end thereof so as to be rotatable with respect to the display unit 3. That is, this imaging unit 22
It can rotate to any position within a range of 180 degrees between the same direction as D21 and the opposite direction (backward direction). A CCD video camera 23 is attached to the imaging unit 22.

A power lamp is provided on the lower main body side of the display unit 3.
PL, battery lamp BL, message lamp ML, other LEDs
Is provided. Reference numeral 40 shown in FIG. 3 is a power switch provided on the left side surface of the main body 2, and reference numeral 25 shown in FIG. 5 is an adjustment ring for adjusting the focus of the CCD video camera 23. Further, FIG.
Reference numeral 26 denotes a lid for covering an opening for mounting an additional memory in the main body 2, and reference numeral 41 denotes a lid 26.
This is a small hole for inserting a pin for removing the lock claw.

FIG. 7 shows an internal configuration of the personal computer 1. As shown in FIG. 7, a CPU (Central Processing Unit) 52, a PC card 53 inserted as needed, and a RAM (Random Access M
emory) 54 and a graphic chip 81 are connected. The internal bus 51 is connected to an external bus 55. The external bus 55 includes a hard disk drive (HDD) 56, an I / O (input / output) controller 57, a keyboard controller 58, a stick pointing device controller 59, and a sound. Chip 60, LC
The D controller 83, the modem 50 and the like are connected.

The CPU 52 is a controller that controls each function, and the PC card 53 is appropriately mounted when an optional function is added.

In the RAM 54, an e-mail program (application program) 54A, an auto-pilot program (application program) 54B, and an OS (basic program) 54C are transferred from the HDD 56 and stored at the time when the activation is completed. You.

The electronic mail program 54A is a program for sending and receiving messages via a network from a communication line such as a telephone line. E-mail program 54
A has an incoming mail acquisition function as a specific function. This incoming mail acquisition function checks the mail server 93 to see if mail addressed to the user (user) has arrived in the mailbox 93A, and if there is mail addressed to the user, obtains the mail. Execute.

The autopilot program 54B includes a plurality of processes (or programs) set in advance.
This is a program that is started and processed sequentially in a preset order.

The OS 54C controls the basic operation of a computer typified by Windows 98 (trademark).

On the other hand, a hard disk drive (HDD) 56 on the external bus 55 side has an e-mail program 56
A, an autopilot program 56B, and an OS 56C are stored. OS in the hard disk drive 56
56C, the auto-pilot program 56B, and the e-mail program 56A are sequentially transferred and stored in the RAM 54 in the course of the startup (boot-up) process.

The I / O controller 57 has a microcontroller 61, and the microcontroller 61 is provided with an I / O interface 62. This microcontroller 61 includes an I / O interface 6
2, the CPU 63, the RAM 64, and the ROM 69 are connected to each other. The RAM 64 includes a key input status register 65 and an LED (light emitting diode) control register 6.
6, a set time register 67 and a register 68 are provided. The set time register 67 is used to start the operation of the start-up sequence control unit 76 at a time (start-up condition) set by the user in advance. The register 68 stores a correspondence between a preset operation key combination (start condition) and an application program to be started. When the stored operation key combination is input by the user, the register 68 stores the correspondence. The activated application program (for example, e-mail) is activated.

The key input status register 65 stores an operation key flag when the programmable power key (PPK) 9 for one-touch operation is pressed. The LED control register 66 controls the lighting of a message lamp ML that indicates the activation state of the application program (e-mail) stored in the register 68. The set time register 67 can arbitrarily set a predetermined time.

A backup battery 74 is connected to the microcontroller 61.
The values of the registers 65, 66, and 67 are retained even when the power of the main unit 2 is turned off.

The ROM 69 in the microcontroller 61 stores a wake-up program 70, a key input monitoring program 71, and an LED control program 72 in advance. The ROM 69 is, for example, an EEPROM (electricall
y erasable and programmable read only memory). This EEPROM is also called a flash memory. Further, an RTC (Real-Time Clock) 75 that constantly counts the current time is connected to the microcontroller 61.

The wake-up program 70 in the ROM 69 checks, based on the current time data supplied from the RTC 75, whether or not the time set in the set time register 67 has been reached. , A program for starting a predetermined process (or a program). The key input monitoring program 71
This is a program for constantly monitoring whether or not K9 has been pressed by the user. The LED control program 72 is a program for controlling lighting of the message lamp ML.

The ROM 69 further contains a BIOS (Basic Input O
utput System) 73 is written. This BIOS 73
Has a function of activating the OS 56C when the power is turned on, and transmitting and receiving data between various application software and peripheral devices (display, keyboard, hard disk drive, etc.) after the activation.

A keyboard controller 58 connected to the external bus 55 controls an input from the keyboard 4. The stick pointing device controller 59 controls an input of the stick pointing device 5.

The sound chip 60 includes the microphone 24
Or an audio signal is supplied to the built-in speaker 8.

The modem 50 can be connected to a communication network 92 such as the Internet, a mail server 93, and the like via a public telephone line 90 and an Internet service provider 91.

After the image data captured by the CCD video camera 23 is processed by the processing unit 82, the graphic chip 81 connected to the internal bus 51
omed Video) port. The graphic chip 81 includes the processing unit 8
Video data input from the CCD video camera 23 via the VRAM 2 is stored in the built-in VRAM 81, read out as appropriate, and output to the LCD controller 83. The LCD controller 83 outputs the image data supplied from the graphic chip 81 to the LCD 21 for display. The backlight 84 illuminates the LCD 21 from behind.

The power switch 40 is operated when the power is turned on or off. The half-press switch 85 is turned on when the shutter button 10 is half-pressed, and the full-press switch 86 is turned on when the shutter button 10 is fully pressed. The reversing switch 87 is connected to the imaging unit 2
2 is rotated 180 degrees (CCD video camera 23
It is turned on when it is rotated in the direction of imaging the opposite side of the CD 21).

The drive 88 is connected to the external bus 55. The drive 88 includes a magnetic disk 351 (including a floppy disk) and an optical disk 352 (CD-ROM (Compa
ct Disc-Read Only Memory), DVD (Digital Versatile D)
isc)), magneto-optical disk 353 (MD (Mini-Dis
c)) or a semiconductor memory 354 or the like, and the mounted magnetic disk 351 and optical disk 35
2. Magneto-optical disk 353 or semiconductor memory 354
Program or data recorded in the HDD 56 or the R via the external bus 55 or the internal bus 51.
Supply to AM54.

The drive 88 is connected to the modem 50, the HDD 56, or the RAM 5 via the external bus 55 or the internal bus 51.
4 is recorded on the mounted magnetic disk 351, optical disk 352, magneto-optical disk 353, semiconductor memory 354, or the like.

FIG. 8 is a diagram showing functional blocks according to a predetermined program when the personal computer 1 activates a program relating to speech recognition. The speech recognition engine 101 is input from the microphone 24 based on the reading of the kanji stored in the reading kana dictionary database 111 in advance, or the recognition word or grammar stored in the engine recognition word / grammar database 112 in advance. Enter the data corresponding to the user's voice
The data in a predetermined format such as text corresponding to the word spoken by the user is generated and supplied to the voice commander 102.

The speech recognition engine 101 receives data such as a recognized word or grammar from the voice commander 102 and stores it in the reading kana dictionary database 111 or the recognized word / grammar database 112 for the engine.

When data such as a word (text, etc.) corresponding to a predetermined word spoken by the user is supplied from the voice recognition engine 101, the voice commander 102 receives a still image photographing program 103, a still image browsing program 104,
Alternatively, it activates the electronic pet program 105 or transmits a predetermined command (corresponding to a word spoken by the user) to the still image photographing program 103, the still image browsing program 104, or the electronic pet program 105.

When data such as a word (text or the like) corresponding to another predetermined word spoken by the user is supplied from the voice recognition engine 101, the voice commander 102 activates the launcher stored in the launcher setting database 113. The e-mail program 54A, the word processor program 106, or the spreadsheet program 107 is started based on the settings related to the above, and predetermined data such as a mail address is supplied to the e-mail program 54A.

The voice commander 102 has a graphical user interface. Various settings are made by the user through the graphical user interface, and the contents set by the user are classified.
Application program (E-mail program 5
4A, the settings related to the activation of the word processor program 106 or the spreadsheet program 107) in the launcher setting database 113, the settings related to reading of kanji, or commands of the still image photographing program 103, the still image browsing program 104, or the electronic pet program 105 Is stored in the dictionary setting database 114, and the setting relating to the word or grammar to be recognized is stored in the recognition word / grammar database 115, respectively.

When the voice commander 102 causes the voice recognition engine 101 to recognize a voice at a predetermined timing, for example, the voice commander 102 converts the recognition word data and the grammar data stored in the recognition word / grammar database 115 into the voice recognition engine. Send to 101.

The voice recognition engine 101 reads the kana dictionary database 111 for the user based on the data inputted when the OS 54C is started to determine the user.
And the engine recognition word / grammar database 112 is used. The voice commander 102 determines the user based on data input when the OS 54C is started,
The launcher setting database 113, dictionary setting database 114, and recognized word / grammar database 115 for the user are used.

A kana dictionary database 111, an engine recognition word / grammar database 112, a launcher setting database 113, a dictionary setting database 114, and a recognition word / grammar database 115 are generated for each user of the personal computer 1 and recorded on the HDD 56. Is done.

The still image shooting program 103 converts the image input from the CCD video camera 23 into the shutter button 1
Based on a signal corresponding to an operation such as 0, data of a still image is generated and recorded in the HDD 56 as a predetermined file.

The still image browsing program 104 selects a file of the still image recorded by the still image photographing program 103 or allows the user to select the file, and causes the LCD 21 to display the selected still image. Electronic pet program 105
Displays a virtual pet on the LCD 21 and gives an instruction to the virtual pet in response to a user operation.

The word processor program 106 is a program for editing a document composed of characters or figures. The spreadsheet program 107 has a function of executing a predetermined operation on numerical values arranged in a table of a predetermined format, or drawing a graph corresponding to the arranged numerical values.

FIG. 9 is a diagram for explaining the functions of the voice commander 102 in more detail. The UI (user interface) processing unit 123 includes an application communication unit 121,
Engine communication unit 122, voice launcher control unit 124, user dictionary control unit 125, or recognition test processing unit 126
And a predetermined signal from the keyboard 4 or the stick-type pointing device 5 and the like, and the loudness of the voice input through the microphone 24 or the result of voice recognition is displayed in a predetermined window. To be displayed. When activating a predetermined program, the UI processing unit 123 displays a predetermined image on the LCD 21 based on data input from the application communication unit 121 or the audio launcher control unit 124.

The UI processing unit 123 changes the state of the UI processing unit 123 itself based on a signal corresponding to an operation of the keyboard 4 or the stick type pointing device 5, and transmits predetermined data to the application communication unit 121.
Engine communication unit 122, voice launcher control unit 124, user dictionary control unit 125, or recognition test processing unit 126
To supply.

Further, the UI processing unit 123 controls the state of the still image photographing program 103, the still image browsing program 104, and the electronic pet program 105, and the user supplied from the voice recognition engine 101 via the engine communication unit 122. Based on data such as a predetermined text corresponding to a predetermined word spoken by the application communication unit 12
1 or the voice launcher control unit 124 determines whether to transmit a command or activate a predetermined program, and causes the application communication unit 121 or the voice launcher control unit 124 to transmit the command or activate a predetermined program. .

The application communication unit 121 includes a still image photographing program 103, a still image browsing program 104,
Alternatively, the electronic pet program 105 is activated, and communicates with the activated still image photographing program 103, still image browsing program 104, or electronic pet program 105, and the still image photographing program 103, the still image browsing program 104, or the electronic pet is activated. Data indicating each state is received from the program 105.

The application communication unit 121 includes a still image photographing program 103, a still image browsing program 104,
In addition to supplying data indicating the state of the electronic pet program 105 to the engine communication unit 122 or the UI processing unit 123, the engine communication unit 122 or the UI processing unit 123 outputs predetermined data corresponding to a predetermined word spoken by the user. It receives data such as text, data corresponding to a user's operation on the keyboard 4, and the like.

The application communication section 121
Still image shooting program 103, still image viewing program 1
04, the state of the electronic pet program 105, and the voice recognition engine 10 via the engine communication unit 122.
On the basis of data such as a predetermined text corresponding to a predetermined word spoken by the user supplied from 1, any one of the still image photographing program 103, the still image browsing program 104, and the electronic pet program 105 is started. Alternatively, a predetermined command is supplied to any one of the still image photographing program 103, the still image browsing program 104, and the electronic pet program 105.

A still image photographing program 103, a still image browsing program 104, and an electronic pet program 105
Are not focused (when neither is active), the voice commander 102
Cannot execute a command for any one of the still image shooting program 103, the still image browsing program 104, and the electronic pet program 105.

The still image photographing program 103, the still image browsing program 104, and the electronic pet program 105
Are focused (when either is active), the voice commander 102
Is an active still image shooting program 103,
A command for either the still image browsing program 104 or the electronic pet program 105 can be executed.

Such a still image photographing program 103,
A command for a specific one of the still image browsing program 104 and the electronic pet program 105 is referred to as a local command.

The method by which the voice commander 102 specifies a program for transmitting a local command is not limited to focus, but may refer to another state or data.

The engine communication unit 122 reads the data of the recognized word and the data of the grammar from the recognized word / grammar database 115 based on a predetermined method, transmits the data to the voice recognition engine 101, and transmits the data to the voice recognition engine 101. Data such as a predetermined text corresponding to a predetermined word spoken by the user supplied from 101 is received.

The engine communication unit 122 is, for example, as shown in FIG.
Microsoft Speech API (trademark) (hereafter, SAPI
The speech recognition engine 10
1, the data of the recognized word and the data of the grammar stored in the recognized word / grammar database 115 are transmitted. In the data example shown in FIG. 10, the target of speech recognition is <Global>
And <SVCommand>, and <Global> is further (Chan
geWin), (VoiceCommand), where <SVCommand> is a command such as "Help" or "Previous", and <SendMail>
It is described that it also includes the mail command represented by. In the example of the data shown in FIG. 10, the code number of the command “help” is 102, and the word having the pronunciation “dad” is “daddy@test.company.co.jp”.
It is shown that the character string is related.

The speech recognition engine 101 converts the data received from the engine communication unit 122 into data of a predetermined format, and stores the converted data in the reading kana dictionary database 111 or the engine recognition word / grammar database 112.
Based on the data stored in the reading kana dictionary database 111 or the engine recognition word / grammar database 112, a speech recognition process is executed.

The speech recognition engine 101 provides the engine communication unit 122 with a code number (for example, 102), a recognized word or sentence (for example, "mail to dad", etc.) corresponding to a predetermined word spoken by the user. ), And the string associated with the recognized word (eg, “daddy@test.compan
y.co.jp ”).

For example, when the user inputs a voice “mail to dad” into the microphone 24 and the voice recognition engine 101 correctly recognizes the voice, the voice recognition engine 101 outputs 7fffffff (hexadecimal), “ "E-mail to dad" and "daddy@test.company.co.jp" are transmitted to engine communication unit 122.

The engine communication unit 122 converts the received data based on the data received from the speech recognition engine 101 into an application communication unit 121, a UI processing unit 123, a speech launcher control unit 124, a user dictionary control unit 125, or a recognition test. It determines which of the processing units 126 to transmit, and based on the determination, converts the data received from the speech recognition engine 101 into a predetermined method, and selects the selected application communication unit 121, UI processing unit 123, The converted data is supplied to any one of the launcher control unit 124, the user dictionary control unit 125, and the recognition test processing unit 126.

The voice launcher control section 124 displays an application program (e-mail program 54A, word processor program 10
6, or the setting relating to the activation of the spreadsheet program 107) is stored in the launcher setting database 113, and the setting relating to the speech recognition word or grammar stored in the recognition word / grammar database 115 is updated based on the setting.

When the voice launcher control unit 124 receives data related to the launcher from the engine communication unit 122, the voice launcher control unit 124 executes the e-mail program 54
A, activates any one of the word processor program 106 and the spreadsheet program 107 and supplies a mail address and the like to the e-mail program 54A.

The voice commander 102 executes a command to activate any one of the e-mail program 54A, the word processor program 106, and the spreadsheet program 107, regardless of the focus state (no matter which program is active). be able to.

Regardless of the focus state, the program can always be executed. For example, the electronic mail program 54A, the word processor program 10
6 or a command for activating one of the spreadsheet programs 107 is referred to as a global command.

The user dictionary control unit 125 displays a graphical user interface, stores the setting relating to the voice to be recognized input by the user in the dictionary setting database 114, and based on the setting,
The setting related to the word or grammar for speech recognition stored in the recognition word / grammar database 115 is updated.

The recognition test processing unit 126 displays a graphical user interface when the user inputs a command to execute a test to the user dictionary control unit 125, stores the graphical user interface in the dictionary setting database 114, and selects and executes the test. The predetermined one word and the engine communication unit 122
And determines whether the word supplied from the voice recognition engine 101 matches the word indicating the result of the voice recognition, and displays the result of the determination.

When the user inputs a command to execute a test to the user dictionary control unit 125, the recognition test processing unit 126 displays a graphical user interface and outputs a voice via the engine communication unit 122. It is determined whether or not the word indicating the result of the voice recognition supplied from the recognition engine 101 is stored in the dictionary setting database 114 and is included in one or more selected predetermined words. Is displayed.

FIG. 11 is a state transition diagram for explaining a state transition of the UI processing unit 123 corresponding to a predetermined input.
In FIG. 11, a statement enclosed in parentheses indicates a state transition condition (for example, program startup, engine activation completed, etc.), and a statement enclosed in [] is executed along with the state transition. Processing (for example, indicating that the device is running,
Grammar settings).

When the voice commander 102 is activated, the UI
The processing unit 123 causes the LCD 21 to display an image indicating that it is being activated, displays the window of the voice commander 102 shown in FIG.

The voice commander window 151 includes a level gauge 161, a recognition result display section 162, a display switching button 163, a launcher setting button 164, a dictionary management button 165, a help button 166, a minimize button 167,
It has a close button 168, a recognition state display section 169, and a voice input mode switching button 170.

The level gauge 161 is connected to the microphone 2
4 shows the level of the user's voice input via the microphone 4 (the amplitude of the signal output from the microphone 24). The recognition result display unit 162 displays a word or a sentence corresponding to the recognized voice supplied from the engine communication unit 122.

The display switching button 163 is operated when the voice commander window 151 is switched to a small window (not shown). Launcher setting button 164
Is operated when making settings relating to activation of the e-mail program 54A, the word processor program 106, or the spreadsheet program 107.

The dictionary management button 165 is operated when the setting relating to the voice to be recognized is stored in the dictionary setting database 114. The help button 166 is operated when displaying online help on the LCD 21. The minimize button 167 allows the voice commander window 151 to be
The operation is performed when the icon is deleted from D21 and a predetermined icon is displayed on the task tray, for example. Close button 1
Reference numeral 68 is operated when the voice commander 102 is terminated.

The recognition state display section 169 displays the state of the speech recognition engine 101, whether a local command can be used (whether a predetermined program is active or not) and the like. Voice input mode switching button 170
Is operated when switching between the normal recognition mode and the normal recognition mode.

In state 1, the speech recognition engine 101
When the start of the operation fails, the UI processing unit 123 ends.

In state 1, the speech recognition engine 101
Is successfully activated, the UI processing unit 123 transitions to a state 2 waiting for a user operation.

When the close button 168 is clicked in the state 2, the UI processing unit 123 ends the voice commander 102. In state 2, when a key (for example, a control key of the keyboard 4 or the like, hereinafter referred to as a recognition key) assigned by the user is pressed, the UI processing unit 123 changes to a state 3 in which voice input is possible.
Transitions to.

At the time of transition from state 2 to state 3, the UI processing unit 123 sends the still image photographing program 103 and the still image browsing program 10 from the application communication unit 121.
4, and data indicating the active program among the electronic pet programs 105 is received, and the name of the active program is input to the voice commander window 15.
1 is displayed on the recognition state display unit 169. When none of the still image photographing program 103, the still image browsing program 104, or the electronic pet program 105 is active, the UI processing unit 123 displays a message to that effect on the recognition status display unit 169 of the voice commander window 151 (for example, “Gl”).
obal Command ”).

In state 3, a signal corresponding to the voice input by the user from the microphone 24 is supplied to the voice recognition engine 101, and data corresponding to the level of the signal corresponding to the voice supplied to the voice recognition engine 101 is output. , The UI processing unit 123 via the engine communication unit 122
Supplied to In state 3, the UI processing unit 123
Based on the data corresponding to the level of the signal corresponding to the voice, the level gauge 16 of the voice commander window 151 is displayed.
Update the display of 1.

In the state 3, when the voice recognition engine 101 recognizes a voice, the UI processing unit 123 receives data such as a recognized word or sentence from the voice recognition engine 101, and displays the recognition result in the voice commander window 151. The display unit 162 displays the recognized word or sentence.

If the user continues to press the recognition key in state 3, the UI processing unit 123 repeats the processing in state 3.

In state 3, when the user releases the recognition key, the UI processing unit 123 sends the code number and the recognized word supplied from the voice recognition engine 101 to the application communication unit 121 or the voice launcher control unit 124. Alternatively, a predetermined operation (for example, activation of the e-mail program 54A) corresponding to the sentence and character string data related to the recognized word is requested.

At this time, the application communication unit 121
Starts one of the still image photographing program 103, the still image browsing program 104, and the electronic pet program 105 in response to a request from the UI processing unit 123, or executes the still image photographing program 103, the still image browsing program 104 Or electronic pet program 105
A predetermined command.

At this time, the voice launcher control unit 124
In response to the request from the UI processing unit 123, the e-mail program 54A, the word processor program 106,
Alternatively, one of the spreadsheet programs 107 is activated, or predetermined data (for example, a mail address) is supplied to the e-mail program 54A.

When the application communication unit 121 or the voice launcher control unit 124 completes a predetermined operation for a predetermined program, the application communication unit 1
21 or the voice launcher control unit 124
The UI processing unit 123 notifies the user of the predetermined program to be operated in accordance with the predetermined program to be operated (described later).
Is displayed on the LCD 21.

An image for recognizing a predetermined program to be operated is displayed on the LCD 21, so that the user can know the result of voice recognition and the operation of the voice commander 102.

At the time of transition from state 3 to state 2, the UI processing unit 123 deletes the display of the recognition state display unit 169.

In the state 2, when the voice input mode switching button 170 is clicked, the UI processing unit 123
Transitions to state 4, which is the always recognition mode.

When transitioning from state 2 to state 4, the UI processing unit 123 sends the still image photographing program 103 and the still image browsing program 10 from the application communication unit 121.
4, and the data indicating the active program among the electronic pet programs 105 are received, and the name of the active program is displayed on the recognition state display unit 169. When none of the still image photographing program 103, the still image browsing program 104, or the electronic pet program 105 is active, the UI processing unit 123 displays the fact (for example, “Global Command”) on the recognition status display unit 169 of the voice commander window 151. Etc.).

In state 4, a signal corresponding to the voice input by the user from the microphone 24 is supplied to the voice recognition engine 101, and data corresponding to the level of the signal corresponding to the voice supplied to the voice recognition engine 101 is output. , The UI processing unit 123 via the engine communication unit 122
Supplied to In state 4, the UI processing unit 123
Based on the data corresponding to the level of the signal corresponding to the voice, the level gauge 16 of the voice commander window 151 is displayed.
Update the display of 1.

When the voice recognition engine 101 recognizes a voice in the state 4, the UI processing unit 123 receives data such as a recognized word or sentence from the voice recognition engine 101, and displays the recognition result in the voice commander window 151. The display unit 162 displays the recognized word or sentence.

In state 4, when the UI processing unit 123 receives data such as a word or a sentence recognized from the speech recognition engine 101, the UI processing unit 123 executes the application communication unit 121 or the voice launcher control unit 124.
Then, a predetermined operation corresponding to the code number, the recognized word or sentence, and the character string data related to the recognized word supplied from the voice recognition engine 101 is requested.

At this time, the application communication unit 121
Starts one of the still image photographing program 103, the still image browsing program 104, and the electronic pet program 105 in response to a request from the UI processing unit 123, or executes the still image photographing program 103, the still image browsing program 104 Or electronic pet program 105
A predetermined command.

At this time, the voice launcher control unit 124
In response to the request from the UI processing unit 123, the e-mail program 54A, the word processor program 106,
Alternatively, one of the spreadsheet programs 107 is activated, or predetermined data (for example, a mail address) is supplied to the e-mail program 54A.

When the application communication unit 121 or the voice launcher control unit 124 completes a predetermined operation for a predetermined program, the application communication unit 1
21 or the voice launcher control unit 124
The UI processing unit 123 notifies the user of the predetermined program to be operated in accordance with the predetermined program to be operated (described later).
Is displayed on the LCD 21.

In state 4, when the voice recognition engine recognizes a predetermined voice irrespective of the operation on the recognition key, the voice commander 2 executes the still image capturing program 10
3. Activate either the still image browsing program 104 or the electronic pet program 105, or execute the still image photographing program 103, the still image browsing program 10
4 or a predetermined command is transmitted to one of the electronic pet programs 105, or one of the electronic mail program 54A, the word processor program 106, and the spreadsheet program 107 is activated, or predetermined data is transmitted to the electronic mail program 54A. Supply.

When the voice input mode switching button 170 is clicked in state 4, the UI processing unit 123
Transitions to state 2.

In state 2, still image shooting program 1
When an operation of photographing a still image is added to 03 (for example, when the shutter button 10 is pressed), the UI processing unit 123 transits to a state 5 for inputting a comment attached to the still image.

In state 5, a signal corresponding to the voice input by the user from the microphone 24 is supplied to the voice recognition engine 101, and data corresponding to the level of the signal corresponding to the voice supplied to the voice recognition engine 101 is output. , The UI processing unit 123 via the engine communication unit 122
Supplied to In state 5, the UI processing unit 123
Based on the data corresponding to the level of the signal corresponding to the voice, the level gauge 16 of the voice commander window 151 is displayed.
Update the display of 1.

In state 5, when the voice recognition engine 101 recognizes a voice, the UI processing unit 123 receives data such as a recognized word or sentence from the voice recognition engine 101 and displays the data on the LCD 21. The recognized word or sentence is displayed in a predetermined dialog or the like corresponding to the image to which the comment is attached.

In the state 5, the UI processing section 123 sends the speech recognition engine 10 to the application communication section 121.
Transmit the recognized word or sentence supplied from 1.
The application communication unit 121 transmits the recognized word or sentence to the still image shooting program 103 and causes the still image shooting program 103 to store the recognized word or sentence as a comment on the image.

In state 5, data indicating an operation of ending input of a comment from the still image photographing program 103 via the application communication unit 121 (for example, the shutter button 10 is released) is transmitted to the UI processing unit 123.
, The UI processing unit 123 transitions to state 2.

In state 2, when dictionary management button 165 of voice commander window 151 is clicked, U
The I processing unit 123 transitions to the dictionary setting state 6 and requests the user dictionary control unit 125 to perform dictionary setting processing.

In state 6, the user dictionary control unit 125
Displays a dictionary setting dialog on the LCD 21 and, based on an operation on the dictionary setting dialog, sets the dictionary setting database 114 and the recognized word / grammar database 11
The setting stored in 5 is updated.

In state 6, when the test button arranged in the dictionary setting dialog is clicked, U
The I processing unit 123 makes a transition to the state 8 in which the voice recognition test is executed, and requests the recognition test processing unit 126 to process the voice recognition test.

The recognition test processing unit 126 causes the LCD 21 to display a dialog of the voice recognition test, and the words recognized by the voice supplied from the voice recognition engine 101 via the engine communication unit 122 are input to the dictionary setting database 11.
Then, a speech recognition test is performed to determine whether the word matches the word registered in No. 4 and the result is displayed.

Alternatively, the recognition test processing unit 126 causes the LCD 21 to display a dialog for a voice recognition test, and the recognized word supplied from the voice recognition engine 101 via the engine communication unit 122 is used by the dictionary setting database 11
Then, a speech recognition test for determining whether the word is included in the words registered in No. 4 is executed, and the result is displayed.

In state 8, when the test button arranged in the dialog for the speech recognition test is clicked, the UI processing unit 123 transitions to state 6.

In state 6, when the close button arranged in the dictionary setting dialog is clicked, U
The I processing unit 123 transitions to state 2.

When the launcher setting button 164 of the voice commander window 151 is clicked in the state 2, the UI processing unit 123 activates the electronic mail program 54A, the word processor program 106, or the spreadsheet program 107 of the voice launcher control unit 124. The state transits to the state 7 in which the setting to perform
Request processing of the program startup setting.

In the state 7, the voice launcher control unit 12
4 displays a launcher setting dialog on the LCD 21 and updates the settings stored in the launcher setting database 113 based on an operation on the launcher setting dialog.

In state 7, when the test button arranged in the launcher setting dialog is clicked, the UI processing unit 123 makes a transition to state 9 in which a speech recognition test is executed, and the recognition test processing unit 126 Request recognition test processing.

The recognition test processing unit 126 displays a dialog for the voice recognition test on the LCD 21 and registers the recognized word or sentence supplied from the voice recognition engine 101 via the engine communication unit 122 in the launcher setting database 113. Then, a speech recognition test is performed to determine whether the recognized word or sentence matches the set grammar, and the result is displayed.

In state 9, when the test button arranged in the dialog for the voice recognition test is clicked, the UI processing unit 123 transitions to state 7.

In state 7, when the close button arranged in the launcher setting dialog is clicked, the UI processing unit 123 transitions to state 2.

FIG. 13 is a state transition diagram for explaining a state transition of the engine communication unit 122 corresponding to a predetermined input. In FIG. 13, statements enclosed in parentheses indicate state transition conditions (for example, program startup, startup end notification, etc.), and statements enclosed in [] are executed along with state transitions. (For example, processing result notification, recognition result transfer, etc.).

When the voice commander 102 is activated, the engine communication unit 122 activates the voice recognition engine 101 and transitions to a state 11 for waiting for the voice recognition engine 101 to be activated. In the state 11, when the initialization fails, the engine communication unit 122 transmits the result to the UI processing unit 123. The UI processing unit 12 that has received the notification that the initialization has failed
3 terminates the voice commander 102, so that when the initialization fails, the engine communication unit 122 terminates.

In state 11, the speech recognition engine 10
When the engine communication unit 122 notifies the UI processing unit 123 of the result of the activation of the first, the engine communication unit 122 transitions to the event waiting state 12.

When the user presses a recognition key (for example, a control key), the UI processing unit 123 transmits data to that effect to the engine communication unit 122. In state 11, the engine communication unit 122 When receiving data indicating that the recognition key has been pressed, the still image photographing program 103, the still image browsing program 104, and the electronic pet program 1 from the application communication unit 121.
The data indicating the active program is received from the recognition word / grammar database 115, and the data indicating the word and the grammar corresponding to the data is read from the recognition word / grammar database 115.

Engine communication section 122 includes data indicating words and grammar corresponding to the active program,
Alternatively, data relating to commands of the still image photographing program 103, the still image browsing program 104, or the electronic pet program 105, etc. are appropriately converted and transmitted to the voice recognition engine 101, and data indicating the result of voice recognition from the voice recognition engine 101. The state transits to the state 13 in which the.

In state 13, the engine communication unit 122
Receives data corresponding to the signal level generated by the voice recognition engine 101 that has received the signal corresponding to the voice input by the user from the microphone 24, and
It is supplied to the processing unit 123. The UI processing unit 123 updates the display of the level gauge 161 of the voice commander window 151 based on the data corresponding to the level of the signal corresponding to the voice.

When a user utters a voice toward the microphone 24, the voice recognition engine 101 detects the voice and outputs data indicating that the voice has been detected to the engine communication unit 12.
Send to 2. In state 13, the engine communication unit 12
2 receives the data indicating that the voice is detected, transfers the data to the UI processing unit 123, and
Transitions to.

In state 14, engine communication unit 122
Receives data corresponding to the signal level generated by the voice recognition engine 101 that has received the signal corresponding to the voice input by the user from the microphone 24, and
It is supplied to the processing unit 123. The UI processing unit 123 updates the display of the level gauge 161 of the voice commander window 151 based on the data corresponding to the level of the signal corresponding to the voice.

In state 14, engine communication unit 122
When receiving data such as a recognized word or sentence from the speech recognition engine 101, the data is supplied to the UI processing unit 123. The UI processing unit 123 updates an image or a character displayed on the LDC 21 based on data such as the recognized word or sentence.

In state 14, engine communication unit 122
The data such as the recognized word or sentence received from the voice recognition engine 101 is stored in the still image photographing program 10.
3. Recognition is performed when any of the still image browsing program 104 or the electronic pet program 105 is activated, or when it is determined that the still image photographing program 103, the still image browsing program 104, or the electronic pet program 105 corresponds to the command. The data such as the selected word or sentence is supplied to the application communication unit 121.

In the state 14, the engine communication unit 122
The data received from the speech recognition engine 101, such as a recognized word or sentence, is stored in the e-mail program 54.
A, when the word processor program 106 or the spreadsheet program 107 is activated, or when it is determined that the data corresponds to the data to be supplied to the e-mail program 54A, the recognized data such as a word or sentence is supplied to the voice launcher control unit 124. I do.

In state 14, when data such as a recognized word or sentence is supplied to the application communication unit 121 or the voice launcher control unit 124, the engine communication unit 122 transitions to state 12.

FIG. 14 is a state transition diagram for explaining a state transition of the application communication unit 121 corresponding to a predetermined input.

When the voice commander 102 is activated, the application communication section 121 makes a transition to the state 21. In the state 21, when the voice commander 102 ends,
The application communication unit 121 ends.

When operating, the application communication unit 121 is always in the state 21, and when the focus of the window changes or a predetermined time has elapsed, the still image photographing program 103, the still image browsing program 104, Alternatively, it checks whether or not the electronic pet program 105 has been activated, or which one is active, and updates predetermined data stored inside the application communication unit 121.

Further, the application communication section 121
When there is a request from the UI processing unit 123 or the engine communication unit 122, the still image photographing program 103, the still image browsing program 104, or the electronic pet program 10
5 indicates whether or not 5 is activated, or which is active (if none is active, data indicating this is included).
Or, it transmits to engine communication section 122.

In state 21, engine communication unit 122
When the application communication unit 121 receives data such as a word or a sentence recognized from the application, the application communication unit 121 activates the still image photographing program 103, the still image browsing program 104, or the electronic pet program 105 according to a procedure shown in a flowchart described later with reference to FIG. To activate any one of the still image photographing program 103, the still image browsing program 104, and the electronic pet program 105, or the still image photographing program 103, the still image browsing program 104, or the electronic pet program 10
5 is supplied with a predetermined command.

In the state 21, an operation for inputting or ending a comment from the still image shooting program 103 (for example, when the shutter button 10 is pressed,
Or release, etc.)
The application communication unit 121 transfers the data to the UI processing unit 123.

Next, the LCD 2 of the personal computer 1
The screen displayed in No. 1 will be described. FIG. 15 is a diagram illustrating a screen displayed on the LCD 21 when the voice commander 102, the voice recognition engine 101, and the still image shooting program 103 are running.

An icon 191 corresponding to the e-mail program 54A and an icon 19 corresponding to the word processor program 106 are provided at predetermined positions on the screen of the LCD 21.
2. Icon 19 corresponding to spreadsheet program 107
3. An audio commander window 151 and a window 194 displayed by the still image shooting program 103 are arranged.

When the stick type pointing device 5 or the like is operated to select the icon 191 and execute the start command (by performing an operation such as selecting from a menu or the like, not shown), the e-mail program 54A is started. . When the icon 192 is selected and the start command is executed, the word processor program 106 is started. When the icon 193 is selected and a start command is executed, the spreadsheet program 107 is started.

A window 194 displayed by the still image shooting program 103 has an image display area 195 for displaying an image picked up by the CCD video camera 23, an effect button 196, and the like.

When the shutter button 10 of the personal computer 1 is pressed from the state shown in FIG. 15, the still image photographing program 103 executes a predetermined format (for example, JPEG (Joint Photo) based on the image captured by the CCD video camera 23.
graphic Experts Group)) and generate a file containing that data,
The generated file is recorded on the HDD 56 and the still image browsing program 104 is started.

When the user inputs a voice (for example, “at Daddy and Hachijojima”) to the microphone 24 while pressing the shutter button 10 of the personal computer 1, U
The I processing unit 123 transits to the state 5, receives the recognized word or sentence supplied from the speech recognition engine 101 via the application communication unit 122, and receives the received word or sentence via the application communication unit 121. The word or sentence is output to the still image shooting program 103.

The still image shooting program 103 pastes the received word or sentence as a comment, for example, at a predetermined position of the header of the still image file.

When the shutter button 10 of the personal computer 1 is released, the UI processing section 123 makes a transition to state 2, and the voice recognition processing ends.

The attachment of the comment to the file of the still image may be executed by the still image browsing program 104.

As shown in FIG. 16, the still image browsing program 104 displays a still image stored in the generated file. The window 201 displayed by the still image browsing program 104 includes an image display area 203 for displaying a still image of the generated file, a comment input dialog 202 for displaying a pasted word or sentence, a button 204 with an e-mail and a name, and the like. Having.

Next, a dialog for registering a word to be attached to a still image file will be described. FIG. 17 shows that when the dictionary management button 165 of the voice commander window 151 is clicked, the user dictionary control unit 125
FIG. 7 is a diagram showing a dictionary management dialog displayed on the screen.

In the dictionary management dialog box, a tab 221 having a mail address and a name clicked when displaying a mail address corresponding to a call name (a word or a sentence to be recognized by the voice recognition engine 101), a personal name corresponding to the call name Tab 222 with the name and name of the person clicked to display the name, and tab 22 with the name of the place clicked to display the name of the place corresponding to the call name
3. A tab 224 labeled with the prefecture and name clicked when displaying the name of the prefecture corresponding to the call name, and a tab 225 labeled with the URL clicked when displaying the URL corresponding to the call name. Are located.

In the example shown in FIG. 17, the tab 222 is clicked, and in the call name display field 226, the call names and the registered words corresponding to the call names that belong to the personal name category and are stored in the dictionary setting database 114 (voice recognition). Is successful, a word entered as a comment) is displayed. For example, the registered word for the nickname "papa" is "papa". When either “Kaori-chan” or “Kaori-in” is recognized, the registered word input as a comment is “Kaori-chan”. Name "
The registered word for “next sister” is “next sister”.

A test button 227 and a registration button 228 for transitioning to state 8 are arranged in the dictionary management dialog.

When the registration button 228 is clicked,
The user dictionary control unit 125 causes the LCD 21 to display a new registration dialog shown in FIG. The new registration dialog box includes a nickname input field 24 for inputting a nickname.
1 and a registered word input field 242 for inputting a registered word. By inputting a predetermined nickname in the nickname input field 241, inputting a registered word corresponding to the nickname in the registered word input field 242, and clicking a button labeled “OK”, the nickname input field 24 is displayed.
Name entered in 1 and registered word input field 2
The registered words (corresponding to the names) in 42 are stored in the dictionary setting database 114.

When the user clicks the button 243 with the test and name in the new registration dialog, the UI processing unit 123
Transits to the state 8, and the recognition test processing unit 126 causes the newly registered word test dialog shown in FIG. 19 to be displayed.

The newly registered word test dialog has a call name input field 241, a message field 252 for displaying a predetermined message, and a test result display field 253. When the user inputs a voice to the microphone 24 (speaks the name to the microphone 24), the recognition test processing unit 126 outputs the voice recognized by the voice recognition engine 101 and the word input to the name input field 241. Is determined, and the result is displayed in the test result display field 253.

In the example of the test result display field 253 shown in FIG. 19, the user inputs “Sister” into the microphone 24 and the voice recognition engine 101
Recognizes "Sister", and recognizes test processor 12
6 is an example displayed when it is determined that “Sister” recognized by the voice recognition engine 101 matches “Sister” entered in the nickname input field 241.

If it is determined that the speech recognized by the speech recognition engine 101 does not match the word entered in the call name input field 241, for example, the recognition test processing unit 126 displays the speech recognition engine 101 in the test result display field 253. Text corresponding to the recognized speech and a message indicating that there is no match (e.g., "N
G ").

When the button 251 named “Testing” in the new registration word test dialog is clicked,
The UI processing unit 123 makes a transition to state 6, and causes the user dictionary control unit 125 to display a dictionary management dialog.

Test button 227 in dictionary management dialog
When is clicked, the UI processing unit 123 transits to the state 8, and the recognition test processing unit 126 displays the registered word test dialog shown in FIG.

The registered word test dialog box has a tab 22
1, tab 222, tab 223, tab 224, tab 22
5, and a message field 272 for displaying a predetermined message in addition to the call name display field 226,
And a test result display field 273.

When the user inputs a voice to the microphone 24 (speaks a name to the microphone 24) while the registered word test dialog is displayed, the recognition test processing unit 126 causes the name display field 226 to be displayed. Is determined whether the word recognized by the speech recognition engine 101 matches any of the words displayed by the speech recognition engine 101, and any one of the words displayed in the call name display field 226 is determined.
If the word matches the recognized word, the matched word displayed in the call name display field 226 is highlighted and displayed.

If it is determined that the word recognized by the speech recognition engine 101 does not match any of the words displayed in the call name display field 226, the recognition test processing unit 126, for example, executes the test result display field 2
A text corresponding to the voice recognized by the voice recognition engine 101 and a message indicating that the text does not match (for example, “NG”) are displayed at 73.

When the button 271 named “Testing” in the registered word test dialog is clicked, the UI
The processing unit 123 makes a transition to state 6, and causes the user dictionary control unit 125 to display a dictionary management dialog.

When the tab 223 of the dictionary management dialog is clicked, the user dictionary control unit 125 assigns the user name to the place name category stored in the dictionary setting database 114 in the call name display field 281 as shown in FIG. , And a registered word (a word input as a comment when speech recognition is successful) corresponding to the nickname. For example, the name “Hachijojima” or “Hachijo”
The registered word for is "Hachijojima". The registered word for the nickname "Tokyo Tower" is "Tokyo Tower". When either “baseball field” or “ball field” is recognized, the registered word input as a comment is “baseball field”.

As shown in FIG. 16, when the still image browsing program 104 displays a predetermined still image and clicks a button 204 with a mail and a name, the still image browsing program 104 When the mail program 54A is not running, the electronic mail program 54A is started.

The still image browsing program 104 causes the electronic mail program 54A to generate a new mail. The still image browsing program 104 attaches a still image file (corresponding to the displayed still image) to the generated new mail.

Furthermore, the still image browsing program 104
Attach the comment stored in the attached still image file to the body of the new mail.

FIG. 22 shows a still image browsing program 104.
Causes the e-mail program 54A to generate a new message, the e-mail program 54A
FIG. 3 is a diagram for explaining a message window 301 displayed on No. 1;

In the attached file display field 312,
The file name of the attached still image file is displayed. The text display field 313 displays the text of a new message pasted from the comment stored in the attached still image file.

When a predetermined e-mail address is entered in the destination field 311 and the transmission window of the message window 301 and the button with the name are clicked, the message displayed in the message window 301 is set in the destination field 311. To your email address
It is transmitted via the Internet 92.

Next, the voice commander 102 activates the e-mail program 54A and the e-mail program 5A.
Processing for causing 4A to generate a mail having a predetermined mail address will be described. When the voice commander 102 is activated and a voice is input to the microphone 24 while the recognition key is being pressed, for example, “mail to sister” is input. Is supplied to the voice commander 102.

As shown in FIG. 23, the UI processing unit 123
Is extended from the voice commander window 151 toward the entire screen (the dotted arrow in the figure (the dotted arrow is
In the direction indicated by an arrow for explanation, which is not actually displayed), a ripple-like image is displayed on the LCD 21.

Next, as shown in FIG. 24, the UI processing unit 1
Reference numeral 23 indicates a convergence from the entire screen toward the icon 191 corresponding to the e-mail program 54A (indicated by a dotted arrow in the figure (a dotted arrow is an arrow for explanation and is not actually displayed). LCD2
1 displays a ripple-like image.

Then, as shown in FIG. 25, the UI processing section 123 highlights the icon 191 corresponding to the electronic mail program 54A. Email program 5
When 4A is not running, the application communication unit 12 of the voice commander 102 transmits the still image browsing program 1
In 04, the e-mail program 54A is activated.

Thereafter, as shown in FIG. 26, the voice launcher control section 124 causes the e-mail program 54A to generate a new message,
A sends a new message to message window 301
To be displayed. The voice launcher control unit 124 sends the e-mail program 54A as a mail address of the new message, the mail corresponding to the “sister” of the “mail to sister” supplied from the voice recognition engine via the engine communication unit 122. Set the address.

In the destination field 311 of the message window 301 for displaying a new message, a mail address (for example, ane@home.zzz) set corresponding to “Sister” of “Mail to Sister” is displayed.

FIG. 27 is a diagram showing an example of the dictionary management dialog when the tab 221 to which the mail address and name of the dictionary management dialog are added is clicked. The nickname display field 351 includes the launcher setting database 1
13, a call name and a registered word corresponding to the call name (a text set as a mail address when speech recognition is successful) belonging to the mail address category are displayed. For example, the registered word for the nickname "papa" is "daddy@nowhere.xxx". When "brother" is recognized, the registered word set as the mail address is "ani@home.yyy". The registered word for the nickname "Sister" is "ane@home.zzz".

Next, the screen display when the voice commander 102 supplies a predetermined command to the still image shooting program 103 will be described. Still image shooting program 103
The effect button 19 of the window 194 displayed by
Clicking on 6 displays an effect setting dialog 371 shown in FIG.

When a predetermined effect is selected from the effect selection field 372 of the effect setting dialog 371, the still image shooting program 103 applies the selected effect to the image captured by the CCD video camera 23.

For example, the effect setting dialog 371
And select "mosaic", the still image shooting program 1
In step 03, a so-called “mosaic” effect is applied to the image captured by the CCD video camera 23, and the image with the “mosaic” effect applied is displayed in the image display area 195.
When “negative / positive inversion” is selected in the effect setting dialog 371, the still image shooting program 103 applies a so-called “mosaic” effect to the image captured by the CCD video camera 23, and displays the image with the “negative / positive inversion” effect applied to the image display area. 195 is displayed.

As shown in FIG. 29, when the still image shooting program 103 is active and the voice commander window 15
1 and the window 194 displayed by the still image photographing program 103 are displayed on the LCD 21, while the recognition key is being pressed, the microphone 24 is input to the microphone 24.
"Mosaic", the voice recognition engine 10
1 supplies data indicating a command corresponding to “mosaic” to the voice commander 102.

As shown in FIG. 30, the UI processing unit 123
Is extended from the voice commander window 151 toward the entire screen (the dotted arrow in the figure (the dotted arrow is
In the direction indicated by an arrow for explanation, which is not actually displayed), a ripple-like image is displayed on the LCD 21.

As shown in FIG. 31, the UI processing unit 123
Is displayed on the LCD 21 so as to converge from the entire screen toward the window 194 (in a direction indicated by a dotted arrow in the drawing (a dotted arrow is an arrow for explanation and is not actually displayed)). Display a ripple-like image.

Then, as shown in FIG. 32, the UI processing section 123 highlights the window 194 displayed by the still image photographing program 103.

The voice commander 102 transmits a command corresponding to “mosaic” to the still image shooting program 103 based on data indicating a command corresponding to “mosaic”. As shown in FIG.
Receives a command corresponding to “mosaic” from the voice commander 102, applies a so-called “mosaic” effect to the image captured by the CCD video camera 23, and displays the image subjected to the “mosaic” effect in the image display area 195. Let it.

Next, the process in which the voice commander 102 activates the word processor program 106 will be described. As shown in FIG. 34, when the voice commander 102 is running, if a voice such as “word processor” is input to the microphone 24 while pressing the recognition key,
The voice recognition engine 101 supplies predetermined data corresponding to “word processor” to the voice commander 102.

As shown in FIG. 35, the UI processing unit 123
Is extended from the voice commander window 151 toward the entire screen (the dotted arrow in the figure (the dotted arrow is
In the direction indicated by an arrow for explanation, which is not actually displayed), a ripple-like image is displayed on the LCD 21.

As shown in FIG. 36, the UI processing unit 123
Are converged from the entire screen toward the icon 192 corresponding to the word processor program 106 (dotted arrows in the figure (dotted arrows are arrows for explanation,
LCD is not displayed)
A ripple-like image is displayed on 21.

Then, as shown in FIG. 37, the UI processing unit 123 highlights the icon 192 corresponding to the word processor program 106.

The voice launcher control section 124 of the voice commander 102 activates the word processor program 106. The word processor program 106 causes the LCD 21 to display a predetermined window 391. A window 391 displayed by the word processor program 106 has a display area 392 for displaying a document file.

For example, a voice such as “word processor” is input to the microphone 24, and the word processor program 106 is input to the voice launcher control unit 124 of the voice commander 102.
Is activated, the voice launcher control unit 124 activates only the word processor program 106. At this time, the word processor program 106 generates a new document and causes the display area 392 to display the new document.

When a voice such as "letter by word processor" is input to the microphone 24 and the word processor program 106 is started by the voice launcher control unit 124 of the voice commander 102, the voice launcher control unit 124 is activated.
Causes the word processor program 106 to open the document file corresponding to the "letter". At this time, as shown in FIG. 39, the word processor program 106 causes the display area 392 to display characters or images corresponding to the document file corresponding to "letter".

Similarly, a voice such as “spreadsheet” is input to the microphone 24 and the voice commander 102 is input.
Spreadsheet control program 107 in the voice launcher control unit 124 of
Is activated, the voice launcher control unit 124 activates only the spreadsheet program 107. Also when the voice launcher control unit 124 activates the spreadsheet program 107, the UI processing unit 123 displays a predetermined ripple-like image,
And highlighting of the icon 193 is executed. FIG.
As shown in (1), the spreadsheet program 107 generates a new document and displays the new document in the display area 412 of the window 411 displayed by the spreadsheet program 107.

For example, when a voice such as “household account book by spreadsheet” is input to the microphone 24 and the spreadsheet control program 107 is started by the voice launcher control unit 124 of the voice commander 102, the voice launcher control unit 124 The document file corresponding to "household account book" is stored in the spreadsheet program 107
To open. At this time, as shown in FIG. 41, the spreadsheet program 107 causes the display area 412 to display characters or images corresponding to the document file corresponding to “household account book”.

Next, a dialog for registering a word corresponding to a program to be started will be described. FIG. 42 shows the launcher setting button 16 of the voice commander window 151.
4, when the voice launcher control unit 113
It is a figure showing the launcher setting dialog displayed on D21.

In the launcher setting dialog box, when a check box with an executable file name is checked, the call name display field 451 displays the call name and the executable file corresponding to the call name stored in the launcher setting database 113. Displays the name (including directory) of. At this time, the execution file corresponding to the name of the execution file displayed in the call name display field 451 is referred to as belonging to the execution file category.

For example, the name of the executable file for the nickname "word processor" is "C: \ ProgramFiles \ ProWord". The name of the executable file for the nickname "spreadsheet" is "C: \ ProgramFiles \ Dasuke". The name of the executable file for the nickname "Draw" is "C: \ ProgramFile"
s1 \ Hanako ”.

The launcher setting dialog box includes a button named “Test” and a button named “Details” and “Name”.
52.

When the check box with the name of the document file is checked, the launcher setting dialog is displayed as shown in FIG.
At 51, a call name and a name of a document file (including a directory) corresponding to the call name stored in the launcher setting database 113 are displayed. For example, the name of the document file for the nickname "household book" is "C: \ document file \ household book". The name of the document file for the nickname "estimate" is "C: \ document file 1 \ estimate". The name of the document file for the nickname "letter" is "C: \ document file 1 \ letter".

At this time, the document file corresponding to the name of the document file displayed in the call name display field 451 is said to belong to the document file category.

When a predetermined call name (for example, “word processor”) displayed in the call name display field 451 and the name of an execution file corresponding to the call name are selected, and a button 452 with details and names is clicked, The voice launcher control unit 113 causes the LCD 21 to display a launcher detailed setting dialog shown in FIG.

In the launcher detailed setting dialog, a voice format [executable file = word processor] corresponding to the nickname (for example, “word processor”) selected in the nickname display field 451 is displayed in the nickname display field 471.
By operating a dictionary reference and a button with a name placed in the launcher detailed setting dialog, the user can use a voice format combining an executable file and a document file, for example, "[executable file = word processor] to select [document] File] ".

The voice launcher control section 124 allocates a predetermined grammar number to the set voice format, and stores it in the launcher setting database 113.

The voice format in which the executable file and the document file set by the user are combined is displayed in the name display field 471. For example, a voice format combining files belonging to the executable file category and files belonging to the document file category shown in FIG.
[Executable file = Word processor] [Document file] "
Has "[document file]" as an argument, so that "[executable file = word processor]
In which the voice corresponding to "[document file]" is recognized (for example, "letter in word processor" and grammar number)
Is supplied, the voice launcher control unit 124 executes, based on the grammar number, an executable file (for example, C: \ Progr) belonging to the executable file category corresponding to “Word Processor”.
amFiles \ ProWord) (that is, by starting the word processing program 106), and copying a document file belonging to the document file category for which the name is set (for example, C: \ document file 1 \ letter corresponding to "letter"). ,
Let the word processor program 106 open it.

Similarly, for example, the sound launcher control unit 12
4, when the voice corresponding to "[document file] in" [executable file = word processor] "is recognized and" housekeeping in word processor "and its grammar number are supplied, the voice launcher control unit 124 sets the grammar number to Open C: \ ProgramFiles \ ProWord, an executable file belonging to the executable file category corresponding to the word processor, and add C: \ Document Files \ Household, which is a document file belonging to the document file category, to a word processor program. Let 106 open.

The launcher control section 124 may cause a predetermined script execution program (not shown) to execute a predetermined script file (a file in which processing procedures such as commands are described). Further, predetermined data (for example, a mail address or image data) may be specified as an argument of the script file. At this time, the launcher setting database 11
Reference numeral 3 stores a script file, data specifying data as an argument of the script file, a grammar number, and the like.

Next, the process of capturing a still image by the CPU 52 executing the voice recognition engine 101, the voice commander 102, and the still image capturing program 103 will be described with reference to the flowchart in FIG. In step S11, the still image shooting program 103 determines whether or not the shutter button 10 has been pressed based on a signal supplied from the pre-press switch 86 via the microcontroller 61, the external bus 55, and the internal bus 51. If it is determined that the shutter button 10 has not been pressed, the process returns to step S11, and the determination process is repeated until the shutter button 10 is pressed.

If it is determined in step S11 that the shutter button 10 has been pressed, the flow advances to step S12, and the still image photographing program 103 activates the still image browsing program 104. In step S13,
The still image shooting program 103 is transmitted to the CCD video camera 23 via the processing unit 82 and the graphic chip 81.
A file of the still image data in a predetermined format is generated based on the signal supplied from the HDD, and the generated still image data file is stored in the HDD 56.

In step S14, the still image shooting program 103 determines whether or not the shutter button 10 has been pressed. If it is determined that the shutter button 10 has been pressed, the process proceeds to step S15, where the voice commander 1
02 is the microphone 24 provided to the speech recognition engine 101.
On the basis of the signal input from the device. In step S16, the voice commander 102
The result of speech recognition processing is received from the speech recognition engine 101.

At step S17, the voice commander 1
02, the result of voice recognition is stored in the still image photographing program 103;
Send to In step S18, the voice commander 1
02 causes the still image photographing program 103 to attach the result of the voice recognition as a comment of the image file.

In step S19, the voice commander 1
02 displays a comment on the still image browsing program 104, and the process ends.

If it is determined in step S14 that the shutter button 10 has been released, the processes in steps S15 to S19 are skipped, and the process ends.

As described above, the user can easily attach a comment to a still image captured by the still image capturing program 103 without operating the keyboard 4 or the like.

Next, the process of command transmission or activation by voice of the CPU 52 executing the voice recognition engine 101 and voice commander 102 will be described with reference to the flowchart of FIG. In step S31, the voice commander 102 determines whether the continuous input mode is selected. If it is determined that the continuous input mode is not selected, the process proceeds to step S32, and determines whether the recognition key is pressed. Is determined.

If it is determined in step S32 that the recognition key has not been pressed, the process returns to step S32, and the voice commander 102 repeats the process of determining whether the recognition key has been pressed until the recognition key is pressed.

If it is determined in step S32 that the recognition key has been pressed, the flow advances to step S33, and the voice commander 102 causes the voice recognition engine 101 to execute voice recognition processing. In step S34, the voice commander 102 receives the result of the voice recognition process from the voice recognition engine 101.

At step S35, voice commander 1
02 displays the result of the speech recognition process received from the speech recognition engine 101.

At step S36, the voice commander 1
02 determines whether or not the recognition key has been released, and if it is determined that the recognition key has been released, the process proceeds to step S37,
After transmitting or starting the command, the process ends.

If it is determined in step S36 that the recognition key has not been released, the flow advances to step S38.
The voice commander 102 determines whether a release key (for example, a shift key) is pressed. If it is determined in step S38 that the release key has been pressed, the process proceeds to step S39, in which the voice commander 102 clears the result of voice recognition, returns to step S33, and repeats the voice recognition process.

If it is determined in step S38 that the release key has not been pressed, step S39 is skipped, the process returns to step S33, and the voice recognition process is repeated.

If it is determined in step S31 that the constant input mode has been selected, the process proceeds to step S40, where the voice commander 102
To execute voice recognition processing. In step S41, the voice commander 102 receives the result of the voice recognition process from the voice recognition engine 101.

At step S42, the voice commander 1
02 displays the result of the speech recognition process received from the speech recognition engine 101. In step S43,
The voice commander 102 executes command transmission or activation, returns to step S31, and repeats command transmission or activation processing.

As described above, by operating the recognition key, the user can confirm the result of the voice recognition, transmit a command or activate a predetermined program.
Execution of processing due to incorrect recognition can be prevented.

Next, step S37 or step S43
A process of starting an application program or transmitting a command of the CPU 52 that executes the voice commander 102, which corresponds to the above, will be described with reference to the flowchart in FIG.
In step S51, the voice commander 102 determines whether or not the result of the voice recognition process received from the voice recognition engine 101 is data for starting a predetermined application program. If it is determined that there is
Proceed to step S52.

At step S52, voice commander 1
02, it is determined whether a predetermined application program corresponding to the result of the voice recognition process received from the voice recognition engine 101 has already been started, and it is determined that the predetermined application program has already been started. Then, the process advances to step S53 to shift the focus of the window to the corresponding application program, and the process ends.

If it is determined in step S52 that the predetermined application program has not been started, the flow advances to step S54, where the voice commander 102 starts the corresponding application program and ends the processing.

If it is determined in step S51 that the data is data for starting a predetermined application program, the flow advances to step S55 to execute the voice commander 102.
Sends the corresponding command to the application program, and the process ends.

As described above, the voice commander 102 can execute command transmission or program start processing in accordance with the state of the program and the data input by voice.

Next, the activation process of the e-mail program 54A of the CPU 52 executing the still image browsing program 104 will be described with reference to the flowchart in FIG. In step S71, the still image browsing program 104
Activate the e-mail program 54A. Step S
At 72, the still image browsing program 104 causes the e-mail program 54A to open a new mail.

At step S73, the still image browsing program 104 attaches the image data to a new mail. In step S74, the still image browsing program 1
In step 04, the comment of the image data is pasted on the body of the new mail, and the process ends.

As described above, the user can easily paste a predetermined image without operating the keyboard 4 and the like.
An e-mail to which a predetermined comment is attached can be generated.

Next, voice commander 102 corresponding to step S37 or step S43 in FIG. 48 is executed.
A process of starting an application program or transmitting data of the CPU 52 will be described with reference to a flowchart of FIG. In step S91, the voice commander 10
2 determines whether or not the argument is included in the result of the speech recognition. If it is determined that the argument is not included in the result of the speech recognition, the process proceeds to step S92 to start the corresponding application program. , The process ends.

If it is determined in step S91 that the argument is included in the speech recognition result, the process proceeds to step S9.
Proceeding to 3, the voice commander 102 determines whether the category of the argument included in the result of voice recognition is a mail address, and determines that the category of the argument included in the result of voice recognition is a mail address. If so, step S94
To start the e-mail program 54A. In step S95, the voice commander 102 opens a new mail in which a mail address corresponding to the argument is set,
The process ends.

If it is determined in step S93 that the category of the argument included in the result of voice recognition is not a mail address, the flow advances to step S96 to execute voice commander 1
In step 02, the document file specified by the argument is opened by the corresponding application program, and the process ends.

[0233] As described above, the voice commander 102 can activate a predetermined program in response to the input voice and cause the program to open a predetermined document file belonging to the document file category.

Next, a CPU for executing the voice commander 102
The image display processing at the time of activation of 52 or command transmission will be described with reference to the flowchart of FIG. In step S101, the voice commander 102 determines whether or not the command is to be transmitted. If it is determined that the command is to be transmitted, the process proceeds to step S102. Display the image.

[0235] In step S103, the voice commander 102 displays a ripple image so as to converge on the window of the application program from the entire screen.

In step S104, voice commander 102 highlights the window of the application program that receives the command, and the process ends.

If it is determined in step S101 that the command has not been transmitted, the application program is started, so the flow advances to step S105, and the voice commander 102 displays a ripple image so as to converge from the entire screen to the window of the application program. Let it.

In step S106, the voice commander 102 displays a ripple image so as to converge on the icon of the application program from the entire screen.

[0239] In step S107, the voice commander 102 highlights the icon of the application program to be started, and the process ends.

As described above, the voice commander 102 displays an image from the voice commander window 151 toward the window of the program for receiving a command, so that the user can be sure that the voice commander 102 is operating. And the operation target of the voice commander 102 can be known.

Further, since the voice commander 102 displays an image from the voice commander window 151 toward the icon of the program to be activated, the user can be sure that the voice commander 102 is operating and that the voice commander 102 is operating. Can be operated.

Next, a CPU for executing the voice commander 102
The speech recognition test process of 52 will be described with reference to the flowchart of FIG. In step S131, the voice commander 102 executes a word recognition test for determining whether or not the predetermined word has been recognized based on the recognition result of the predetermined word supplied from the voice recognition engine 101.

In step S132, the voice commander 102 determines whether a predetermined word has been recognized based on the result of step S131. If it is determined that the predetermined word has been recognized, the process proceeds to step S133. A test for recognizing words included in a predetermined category is performed.

In step S134, the voice commander 102 determines whether a word included in the predetermined category has been recognized. If it is determined that a word included in the predetermined category has been recognized, the process proceeds to step S135. Perform grammar recognition tests when multiple words are involved.

In step S136, voice commander 102 determines whether a grammar including a plurality of words has been recognized. If it is determined that a grammar including a plurality of words has been recognized, the process ends. I do.

If it is determined in step S136 that the grammar including a plurality of words is not recognized, the process proceeds to step S138, in which the voice commander 102 corrects the grammar and returns to step S135 to include the plurality of words. When the grammar recognition test is repeated.

If it is determined in step S134 that a word included in the predetermined category is not recognized, and if it is determined in step S132 that the predetermined word is not recognized, the process proceeds to step S137, where the voice commander 102 Is corrected, the process returns to step S131, and the process is repeated from the word recognition test.

As described above, the voice commander 102 displays the result of the voice recognition test and corrects the word or grammar, so that the predetermined voice can be reliably recognized.

A description will now be given, with reference to the flowchart of FIG. 52, of a process of testing a registered name by the CPU 52 executing the voice commander 102, which corresponds to the word recognition test in step S131. In step S151, the voice commander 102 receives the result of voice recognition from the voice recognition engine 101.

In step S152, the voice commander 102 determines whether or not the name matches the voice recognition result. If it is determined that the name matches the voice recognition result, the process advances to step S153 to display the test result. The fact that the recognition has been correctly made is displayed in the field 253, and the process ends.

If it is determined in step S152 that the speech recognition result does not match the name, the process proceeds to step S152.
Proceeding to 154, the voice commander 102 displays in the test result display field 253 that the recognition has not been correctly performed, and the process ends.

CPU 52 executing voice commander 102 corresponding to the category recognition test in step S133
The process of the test of the name of the category is described with reference to the flowchart of FIG. In step S171, the voice commander 102 receives the result of voice recognition from the voice recognition engine 101.

In step S172, voice commander 102 determines whether or not a name matching the result of voice recognition has been registered. If it is determined that a name matching the result of voice recognition has been registered, Step S17
3 and highlighting the call name that is displayed in the call name display field 226 and that matches the result of voice recognition,
The process ends.

If it is determined in step S172 that the name matching the result of the voice recognition is not registered, the process proceeds to step S174, where the voice commander 102
In the test result display field 273, a message indicating that the name corresponding to the voice recognition result is not registered is displayed, and the process ends.

Next, a CPU for executing the voice commander 102
A series of processes from starting the still image photographing program 103 to capturing a still image and attaching a comment to the still image will be described with reference to flowcharts in FIGS. 54 and 55. In step S211, the engine communication unit 122 activates the speech recognition engine 101 and makes a transition to state 11. In step S251,
The UI processing unit 123 executes a display indicating that the engine is running, and makes a transition to state 1. In step S301, the application communication unit 121 transitions to the state 21.

In step S212, the engine communication unit 122 receives data indicating that the activation has succeeded from the speech recognition engine 101, and transmits data indicating that the activation of the speech recognition engine 101 has succeeded to the UI processing unit 123. I do. In step S252, the UI processing unit 123
Receives data indicating that the activation of the speech recognition engine 101 has been successful, and transitions to state 2.

In step S253, the UI processing unit 1
23 causes the LCD 21 to display an image or a sentence indicating that the speech recognition engine 101 has been started.

At step S254, the UI processing unit 1
23 detects whether or not the control key which is the recognition key is pressed. When the control key serving as the recognition key is pressed, the process proceeds to step S255,
The I processing unit 123 provides the application communication unit 121 with:
Still image shooting program 103, still image viewing program 1
Request transmission of data indicating which of the electronic program 04 and the electronic pet program 105 is focused (whether any program is active or none of them is active). In step S302, the application communication unit 121
Receives a request from the application communication unit 121 to transmit data indicating which of the still image photographing program 103, the still image browsing program 104, and the electronic pet program 105 is in focus.

In step S303, application communication section 121 transmits data indicating which of still image photographing program 103, still image browsing program 104, and electronic pet program 105 is in focus. In step S256, the UI processing unit 123 sends the
Still image shooting program 103, still image viewing program 1
04, or data indicating which of the electronic pet programs 105 is in focus. In step S257, the still image shooting program 10
3. Upon receiving data indicating that neither the still image browsing program 104 nor the electronic pet program 105 is focused, the UI processing unit 123
Is the recognition status display section 1 of the voice commander window 151.
69, a character indicating that only a global command is accepted (for example, “(Global Command)
d))).

At step S258, the UI processing unit 1
23 receives a voice from the engine communication unit 122 (that is,
Send a request to enable voice input) and state 3
Transitions to. In step S213, the engine communication unit 122 receives the request for voice reception. Step S2
In 14, the engine communication unit 122 determines which of the still image photographing program 103, the still image browsing program 104, and the electronic pet program 105 is focused on the application communication unit 121 (which program is active , Or if none of them are active).

In step S 305, the application communication unit 121 sends the still image photographing program 103, the still image browsing program 104,
Alternatively, data indicating which of the electronic pet programs 105 is in focus is transmitted.

[0262] In step S215, the engine communication unit 122 sends the still image photographing program 103, the still image browsing program 104, or the electronic pet program 105.
Receives data indicating which of the focuses is on. In step S216, the engine communication unit 122 reads the currently acceptable word and grammar data from the recognized word / grammar database 115, transmits the read data to the speech recognition engine 101, and transitions to state 13.

When the user starts inputting voice to the microphone 24, the process proceeds to step S217, where the engine communication unit 122 receives data indicating voice detection from the voice recognition engine 101, and transitions to state 14.

In step S218, engine communication section 122 receives data corresponding to the level of a signal (indicating the volume of the voice) corresponding to the voice from voice recognition engine 101. In step S219, the engine communication unit 122 transmits data corresponding to the level of the signal corresponding to the voice to the UI processing unit 123. Step S259
In, the UI processing unit 123 receives data corresponding to the level of a signal corresponding to audio. Step S260
In, the UI processing unit 123 updates the display of the level gauge 161 of the voice commander window 151 based on the data corresponding to the level of the signal corresponding to the received voice.

In step S 220, engine communication section 122 receives data corresponding to the level of a signal (indicating the volume of the voice) corresponding to the voice from voice recognition engine 101. In step S221, the engine communication unit 122 transmits data corresponding to the level of the signal corresponding to the voice to the UI processing unit 123. Step S261
In, the UI processing unit 123 receives data corresponding to the level of a signal corresponding to audio. Step S262
In, the UI processing unit 123 updates the display of the level gauge 161 of the voice commander window 151 based on the data corresponding to the level of the signal corresponding to the received voice.

The level of a signal corresponding to voice is received from the voice recognition engine 101, and the voice commander window 1 is received.
The process of updating the display of the level gauge 161 of 51 is repeated as needed in a state where voice reception is possible.

[0267] In step S222, the engine communication unit 122 transmits a code number (for example, 10) corresponding to a predetermined word spoken by the sound user from the speech recognition engine 101.
6), a recognized word or sentence (eg, “photographing”), and character string data related to the recognized word. In step S223, the engine communication unit 122 transmits the code number, the recognized word or sentence, and character string data related to the recognized word to the UI processing unit 1.
23. In step S263, the UI processing unit 123 receives the code number, the recognized word or sentence, and the data of the character string related to the recognized word.
In step S264, the UI processing unit 123 displays the recognized word or sentence on the recognition result display unit 162 of the voice commander window 151.

In step S224, the engine communication unit 122 determines that the data received also should be transmitted to the application communication unit 121 based on the code number, and determines the code number, the recognized word or sentence, and the recognized word. The data of the related character string is transmitted to the application communication unit 121, and the state transits to the state 13. In step S306, the application communication unit 121
Receive code number, recognized word or sentence, and character string data related to the recognized word.

At step S265, the UI processing unit 1
23 detects whether the control key which is the recognition key is released. When the control key which is the recognition key is released, the process proceeds to step S266, and the UI
The processing unit 123 requests the engine communication unit 122 to end the voice reception, and the application communication unit 121
In step 306, the application communication unit 1
A predetermined process (for example, the still image photographing program 1) based on the code number, the recognized word or sentence, and the data of the character string related to the recognized word
03 is started), and the state transits to the state 2.

In step S225, engine communication unit 122 receives a request from UI processing unit 123 to end voice reception. In step S226, the engine communication unit 122 causes the speech recognition engine 101 to stop accepting speech, and makes a transition to state 12.

In step S307, the application communication section 121 receives a request from the UI processing section 123. In step S308, the application communication unit 121 activates the still image shooting program 103.

In step S 309, the application communication section 121 receives a comment input request corresponding to the depression of the shutter button 10 from the still image shooting program 103. In step S310, the application communication unit 121 transmits a comment input request to the UI processing unit 123. Step S267
, The UI processing unit 123 receives a request for comment input. In step S268, the UI processing unit 12
3 transmits a comment input request to the engine communication unit 122, and transitions to state 5. In step S227, engine communication unit 122 receives a request for comment input.

In step S 228, engine communication section 122 reads the currently acceptable word and grammar data from recognized word / grammar database 115, transmits the read data to speech recognition engine 101, and transitions to state 13. .

When the user starts inputting voice to the microphone 24, the process proceeds to step S229, where the engine communication unit 122 receives data indicating voice detection from the voice recognition engine 101, and transitions to state 14.

In step S230, engine communication section 122 receives data corresponding to the level of the signal corresponding to the voice (indicating the volume of the voice) from voice recognition engine 101. In step S231, the engine communication unit 122 transmits data corresponding to the level of the signal corresponding to the voice to the UI processing unit 123. Step S269
In, the UI processing unit 123 receives data corresponding to the level of a signal corresponding to audio. Step S270
In, the UI processing unit 123 updates the display of the level gauge 161 of the voice commander window 151 based on the data corresponding to the level of the signal corresponding to the received voice.

[0276] In step S232, the engine communication unit 122 transmits a code number (for example, 10) corresponding to a predetermined word spoken by the sound user from the speech recognition engine 101.
7), a recognized word or sentence (eg, “Dad and Hachijojima”, etc.), and character string data related to the recognized word. In step S233, the engine communication unit 122 stores the code number, the recognized word or sentence, and the data of the character string related to the recognized word in the U
It is transmitted to the I processing unit 123. In step S271, the UI processing unit 123 receives the code number, the recognized word or sentence, and the data of the character string related to the recognized word. In step S272, the UI processing unit 1
23 displays the recognized word or sentence on the recognition result display section 162 of the voice commander window 151.

In step S234, the engine communication unit 122 determines that the received data should also be transmitted to the application communication unit 121 based on the code number, and determines the code number, the recognized word or sentence, and the recognized word. The data of the related character string is transmitted to the application communication unit 121, and the state transits to the state 13. In step S311, the application communication unit 121
Receive code number, recognized word or sentence, and character string data related to the recognized word.

In step S312, application communication section 121 transmits the recognized word or sentence as a comment to still image shooting program 103, and causes the still image shooting program 103 to attach the recognized word or sentence as a comment.

The processes of steps S232 to S312 are repeatedly executed when the shutter button 10 is not released and the voice of the user is recognized by the voice recognition engine 101.

In step S313, the application communication unit 121 responds to an operation such as releasing the shutter button 10 from the still image photographing program 103.
Receives a request to end comment input. Step S31
In 4, the application communication unit 121 transmits a request to end the comment input to the UI processing unit 123. In step S273, the UI processing unit 123 receives a request to end comment input. In step S274, the UI processing unit 123 transmits a request to end the comment input to the engine communication unit 122, and makes a transition to state 2. In step S225, the engine communication unit 122
Receives a request to end comment input. Step S
In 236, the engine communication unit 122 causes the speech recognition engine 101 to end accepting speech, transitions to state 12, and ends the processing.

As described above, the voice commander 102 responds to the input of voice without complicated operation on the keyboard 4 or the stick-type pointing device 5,
The user can start the still image shooting program 103 and input a comment to the still image captured by the still image shooting program 103.

Note that the speech recognition engine 101 may have a function of recognizing not only a registered word or sentence but also an arbitrary word or sentence (a so-called dictation function).

The above-described series of processes can be executed by hardware, but can also be executed by software. When a series of processing is executed by software, a program constituting the software can execute various functions by installing a computer built into dedicated hardware or installing various programs. It is installed from a program storage medium to a possible general-purpose personal computer or the like.

As shown in FIG. 7, a program storage medium for storing a program installed in a computer and made executable by the computer includes a magnetic disk 95 (including a floppy disk) and an optical disk 96 (CD-ROM ( Compact Disc-Read Only Memory), DVD (D
digital Versatile Disc), magneto-optical disc 97
(Including MD (Mini-Disc)) or semiconductor memory 9
8 or a ROM 69 for temporarily or permanently storing a program, a hard disk 56, or the like. The storage of the program in the program storage medium can be performed by a router,
This is performed using a wired or wireless communication medium such as a local area network, the Internet 92, or a digital satellite broadcast (not shown) via an interface such as the modem 50.

[0285] In this specification, the step of describing a program stored in a program storage medium may be performed not only in chronological order but also in chronological order in the order described. , And also includes processes executed in parallel or individually.

[0286] In this specification, the system is
It represents the entire device composed of a plurality of devices.

[0287]

According to the information processing apparatus according to the first aspect, the information processing method according to the third aspect, and the program storage medium according to the fourth aspect, a word corresponding to the voice is recognized by recognizing the voice. Is generated, a command of a predetermined program is stored in association with the word, and a command corresponding to the generated word is transmitted to the program.
You can easily operate the program.

[Brief description of the drawings]

FIG. 1 is an external perspective view of a personal computer 1. FIG.

FIG. 2 is a plan view of the personal computer 1.

FIG. 3 is a left side view of the personal computer 1.

FIG. 4 is a right side view of the personal computer 1.

FIG. 5 is a front view of the personal computer 1.

FIG. 6 is a bottom view of the personal computer 1.

FIG. 7 is a block diagram showing a configuration of the personal computer 1.

FIG. 8 is a diagram showing functional blocks of the personal computer 1.

FIG. 9 is a diagram showing functional blocks of the personal computer 1.

FIG. 10 is a diagram illustrating SAPI.

11 is a state transition diagram of a UI processing unit 123. FIG.

FIG. 12 is a diagram showing a voice commander window 151.

13 is a state transition diagram of the engine communication unit 122. FIG.

FIG. 14 is a state transition diagram of the application communication unit 121.

FIG. 15 is a diagram illustrating a screen displayed on the LCD 21.

FIG. 16 is a diagram illustrating a screen displayed on the LCD 21.

FIG. 17 is a diagram illustrating a screen displayed on the LCD 21.

FIG. 18 is a diagram illustrating a screen displayed on the LCD 21.

FIG. 19 is a diagram illustrating a screen displayed on the LCD 21.

FIG. 20 is a diagram illustrating a screen displayed on the LCD 21.

FIG. 21 is a diagram illustrating a screen displayed on the LCD 21.

FIG. 22 is a diagram illustrating a screen displayed on the LCD 21.

FIG. 23 is a diagram illustrating a screen displayed on the LCD 21.

FIG. 24 is a diagram illustrating a screen displayed on the LCD 21.

FIG. 25 is a diagram illustrating a screen displayed on the LCD 21.

FIG. 26 is a diagram illustrating a screen displayed on the LCD 21.

FIG. 27 is a diagram illustrating a screen displayed on the LCD 21.

FIG. 28 is a diagram illustrating a screen displayed on the LCD 21.

FIG. 29 is a diagram illustrating a screen displayed on the LCD 21.

FIG. 30 is a diagram illustrating a screen displayed on the LCD 21.

FIG. 31 is a diagram illustrating a screen displayed on the LCD 21.

FIG. 32 is a diagram illustrating a screen displayed on the LCD 21.

FIG. 33 is a diagram illustrating a screen displayed on the LCD 21.

FIG. 34 is a diagram illustrating a screen displayed on LCD 21.

FIG. 35 is a diagram illustrating a screen displayed on the LCD 21.

36 is a diagram illustrating a screen displayed on LCD 21. FIG.

FIG. 37 is a diagram illustrating a screen displayed on the LCD 21.

FIG. 38 is a diagram illustrating a screen displayed on the LCD 21.

FIG. 39 is a diagram illustrating a screen displayed on the LCD 21.

40 is a diagram illustrating a screen displayed on LCD 21. FIG.

FIG. 41 is a diagram illustrating a screen displayed on LCD 21.

FIG. 42 is a diagram illustrating a screen displayed on LCD 21.

FIG. 43 is a diagram illustrating a screen displayed on LCD 21.

FIG. 44 is a diagram illustrating a screen displayed on LCD.

FIG. 45 is a flowchart illustrating a process of capturing a still image.

FIG. 46 is a flowchart illustrating processing for command transmission or activation by voice.

FIG. 47 is a flowchart illustrating processing for starting an application program or transmitting a command.

FIG. 48 is a flowchart illustrating a process of activating the e-mail program 54A.

FIG. 49 is a flowchart illustrating a process of starting an application program or transmitting data.

FIG. 50 is a flowchart illustrating a process of displaying an image when starting or transmitting a command.

FIG. 51 is a flowchart illustrating a speech recognition test process.

FIG. 52 is a flowchart illustrating a test process for a registered name to be registered;

FIG. 53 is a flowchart illustrating a process of testing a category name;

FIG. 54: Activating the still image shooting program 103
9 is a flowchart illustrating a series of processing from capturing a still image to attaching a comment to the still image.

FIG. 55 activates the still image shooting program 103,
9 is a flowchart illustrating a series of processing from capturing a still image to attaching a comment to the still image.

[Explanation of symbols]

1 personal computer, 4 keyboard, 1
0 shutter button, 21 LCD, 23 CCD video camera, 24 microphone, 52 CPU, 59
ROM, 54 RAM, 84A E-mail program, 86 Front switch, 88 drive, 9
2 Internet, 95 Magnetic disk, 96
Optical disk, 97 magneto-optical disk, 98 semiconductor memory, 101 voice recognition engine, 102 voice commander, 103 still image photographing program, 104
Still image browsing program, 111 reading kana dictionary database, 112 recognition word / grammar database for engine, 113 launcher setting database, 114
Dictionary setting database, 115 recognized word database, 121 application communication unit, 122
Engine communication unit, 123 UI processing unit, 124 voice launcher control unit, 125 user dictionary control unit, 12
6 Recognition test processing unit

 ──────────────────────────────────────────────────続 き Continued on the front page (72) Inventor Shuji Yonekura 6-7-35 Kita-Shinagawa, Shinagawa-ku, Tokyo Inside Sony Corporation (72) Inventor Satoshi Fujimura 6-35, Kita-Shinagawa, Shinagawa-ku, Tokyo Inside Sony Corporation (72) Inventor Takashi Sasai 6-7-35 Kita-Shinagawa, Shinagawa-ku, Tokyo Sony Inside (72) Inventor Naoki Fujisawa 7-35, Kita-Shinagawa, Shinagawa-ku, Tokyo Sony (72) Inventor Junji Oi 6-7-35 Kita-Shinagawa, Shinagawa-ku, Tokyo Sony Corporation F-term (reference) 5D015 KK01 KK03

Claims (4)

[Claims] A generating means for recognizing a voice and generating a word corresponding to the voice; a storing means for storing a command of a predetermined program in association with the word; Transmitting means for transmitting the command corresponding to the word generated by the command. 2. The apparatus according to claim 1, further comprising a selection unit that selects the program from operating programs, wherein the transmission unit transmits the command to the program selected by the selection unit. Item 2. The information processing device according to item 1. A generating step of recognizing a voice and generating a word corresponding to the voice; a storing step of storing a command of a predetermined program in association with the word; A transmitting step of transmitting the command corresponding to the word generated in the processing of (a). 4. A generating step of recognizing a voice and generating a word corresponding to the voice; a storing step of storing a command of a predetermined program in association with the word; A transmission step of transmitting the command corresponding to the word generated in the process of (b).