JP3382958B2 - Control device and method for electronic equipment - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a control device and method of an electronic device such as a VTR (video tape recorder) for performing recording reservation and direct operation control of the electronic device.

[0002]

2. Description of the Related Art In recent years, various household electronic devices such as VTRs (video tape recorders) have become multifunctional, and a large number of switches for setting and controlling various operation modes are provided on the device body. The operation is complicated, or the operation procedure is complicated even with a small number of switches. For this reason, the fact is that few people can effectively use the timer function for making a recording reservation of the VTR, for example. Various methods have been considered for facilitating such an operation. For example, a so-called bar code reading device is used to simplify the input operation, and a natural language, that is, human words, is input from a keyboard or a microphone. Attempts have been made to perform operation mode setting control by inputting commands and various information in a form. In the method of reading the bar code, the bar code indicating various information of the program to be reserved is required in advance, and thus the degree of freedom is low and the use range is limited. Therefore, the bar code is input in the natural language. The method is considered to be influential in the future.

Here, for example, JP-A-61-272239
In the publication, in a voice input operation device capable of voice-inputting an operation command for designating an operation mode of an information recording / reproducing device such as a VTR by a human voice, the data processing means includes status data of the information recording / reproducing device. , A voice input device capable of stably controlling a VTR or the like by generating a control command for the VTR or the like is disclosed. Further, in Japanese Patent Laid-Open No. 2-133803, a natural language expressed by a combination of words forming a natural language representing a plurality of control commands of various electronic devices to be controlled is analyzed and a control command is given to the control target. There is disclosed a control data input device capable of performing control by natural language input by giving.

[0004]

According to these techniques, it is possible to control the setting of the operation mode of an electronic device such as a VTR by inputting voice in a natural language form via a microphone or the like. There is room for improvement in the response to the input, for example, in order to deal with the case where the reservation contents cannot be completely corrected or reserved only by displaying the reservation contents as a data string of numbers or characters on the display panel etc. Prior knowledge is required and it is hard to say that anyone can operate it easily. In addition, when the displayed contents are a list of data such as numbers, many people may cause a rejection reaction and may be conscious of disappointment, and further improvement on the man-machine interface is currently required. . Further, when voice input is performed in the natural language state as described above, voice input consisting of a plurality of discrete words is continuously input. Therefore, in the above-described conventional voice input device, it is difficult to determine the clear start timing and end timing of each word during the voice recognition process for analyzing the meaning of each word of the input voice signal. Therefore, for example, unnecessary voice recognition is performed for noise other than the word voice, and it takes time for the voice recognition.

Therefore, the present invention has been proposed in view of the above situation, and not only can anyone easily perform the control operation, but also the operation is easy and familiar to the human operator. An object of the present invention is to provide a control device and method for an electronic device in which a machine interface is realized.

[0006]

An electronic equipment control device of the present invention has been proposed to achieve the above-mentioned object, and an input state is designated in the electronic equipment control device for controlling an electronic equipment. Switching means for extracting the word information representing a command or various information from the input signal, an animation character generating means for generating a video signal of an animation character serving as a speaker of the message, and a message signal input Voice synthesizing means for synthesizing the voice signal of the message, and voice output means for outputting the voice signal from the voice synthesizing means as voice.
At least, based on the word information extracted by the extraction means and the information accumulated in the habit learning database, a motion control signal for controlling the motion of the animation character of the animation character generation means, and the voice synthesis means. And a control means for controlling a message signal for instructing a message voice to be synthesized. The control means specifies and controls the input state by turning on / off the switch means, and responds to turning on of the switch means. The operation of the animation character of the animation character generation means is controlled to be switched, and it is determined whether or not the word information extracted by the extraction means lacks information necessary for controlling the electronic device. If there is a shortage, the habit learning database contains the necessary information. Whether determined, it is characterized in that for controlling the message signal to the necessary information to confirm the audio output if it contains the necessary information to the habit learning database. Further, the control method of the electronic device of the present invention is proposed in order to achieve the above object, in the control method of the electronic device for controlling the electronic device, a switch step for designating an input state, An extraction step of extracting word information representing a command or various information from an input signal, an animation character generation step of generating a video signal of an animation character serving as a speaker of a message, and an audio signal of the message according to a message signal input. A voice synthesizing step of synthesizing, a voice output step of outputting the voice signal from the voice synthesizing step as a voice, and based on at least the word information extracted in the extracting step and the information accumulated in the habit learning database. , Motion control signals that control the motion of the animation character in the animation character generation process And a control step of controlling a message signal instructing a message voice which is desired to be synthesized by the speech synthesis step, in the control step, on with the switch step /
While controlling the input state by turning it off, switching control of the motion of the animation character according to the turning on in the switch step, and information necessary for controlling the electronic device is included in the word information extracted by the extracting step. It is determined whether or not the necessary information is included in the habit learning database when the necessary information is insufficient, and the necessary information is included in the habit learning database. When included, the above-mentioned message signal is controlled so that the necessary information is output by voice and confirmed.

[0007]

According to the present invention, a switch for designating the input state is provided, and the processing operation after the extraction of the word information from the input signal is switched according to the ON / OFF state of the switch, thereby separating the input state. It becomes possible to attach them, and by processing the separated input states as a group, more accurate processing is possible.

[0008]

Embodiments of the present invention will be described below with reference to the drawings. As shown in FIG. 1, the control device for an electronic device according to the present embodiment is a control device for an electronic device that specifies and controls an operation mode of an electronic device such as a VTR (video tape recorder) 40. In the present embodiment, the VTR 40 is used as the electronic device, and the VTR 40 is controlled by various control modes such as recording, reproducing, fast-forwarding, rewinding, and information control such as recording reservation. To do.

That is, in the apparatus of the present embodiment shown in FIG. 1, as a voice input means for inputting a command and various information for designating the operation mode by voice, for example, the same shape as a so-called telephone handset Handset 1
0 is set. The transmitter 11 of the handset 10 is provided with an acoustic-electric conversion element that converts an input voice into an electrical signal and outputs the electrical signal, and a voice input state is switched and designated near the transmitter 11. For example, a switch (press talk switch) 12 is provided to attach input delimiters and the like. The output signal from the transmitter 11 of the handset 10 is sent to the voice recognition circuit 13 and undergoes signal processing to recognize the above-mentioned command and various information. The output signal from the switch 12 is sent to the switch state detection unit 14 to turn on / off.
The off state is detected. These voice recognition circuits 13
The switch state detector 14 is connected to the control circuit 15. In addition to this, the control circuit 15 is provided with an animation character generation circuit 16 for outputting a video signal of an animation character serving as a speaker of the message, and a voice signal of the message according to a message signal input, for example, a so-called regular voice synthesis method. A voice synthesizing circuit 19 to be used for synthesis and a VTR controller 18 for designating and controlling the operation mode of the VTR 40 are connected. The video signal from the character generation circuit 16 is sent to the superimposer 17 and superposed on the video signal from the VTR 40, and a cathode ray tube (C
RT) is sent to the display device 30. Speech synthesis circuit 19
The audio signal from is sent to the speaker 20 as an audio output means and is output as audio. The speaker 20 and the CRT display device 30 may be collectively configured as a television receiver, for example.

The control circuit 15 controls the operation mode designation control signal for designating and controlling the operation mode of the VTR 40 in accordance with at least the output signal from the voice recognition circuit 13 and the action of the animation character AC of the animation character generation circuit 16. It outputs an operation control signal for controlling and a message signal for instructing a message voice to be synthesized by the voice synthesis circuit 19, and has a CPU such as a microprocessor. The control circuit 15 controls one of a plurality of control commands, which is suitable for the current operating state of the VTR 40, according to the command for controlling the operation mode designation recognized by the voice recognition circuit 13. The command is selected and output. Details of this selection process and the operation mode of the VTR 40 will be described later. Further, the control circuit 15 outputs a message signal instructing an optimal message as a response content according to the voice input content (further according to the current state),
This message signal is sent to the animation character generation circuit 16 and the voice synthesis circuit 19.

In FIG. 1, the transmitter 11 of the handset 10 converts a voice uttered by a person (for example, an operator who operates the VTR 40 or the like, that is, a user) into an electric voice signal. The voice uttered by the operator is, for example, “play”, “stop”, “record”, for directly controlling the various operation modes of the VTR 40.
The pronunciation of words of commands such as "pause", "slow", "fast forward", "rewind", etc., and various types of information, such as the day of the week for recording reservations ("Sunday" to "Saturday",
"Everyday", "Weekly" etc.), channel ("1st channel" to "12th channel", etc.), start / end time ("AM", "PM", "0:00" to "12:00", etc.) And the like can be mentioned as a voice. Further, in the present embodiment, in addition to the above-mentioned words, more human words such as "Ooi", "more good", and "useless" can be input by voice.

The press talk switch 12 for designating the voice input state arranged on the handset 10 is for turning on / off by the operator to instruct a break for a word pronounced by the operator. Is. That is, the press talk switch 12 is provided to separate the processing unit of the voice recognition in the voice recognition circuit 13 from the input voice signal composed of a sentence composed of a plurality of continuous discrete words. The output from the press talk switch 12 is sent to the switch state detection unit 14 provided in the voice recognition circuit 13. The switch state detecting unit 14 is turned on / off at present depending on the output signal from the press talk switch 12.
A state indicating signal indicating the off state is formed. The state instruction signal is, for example, state 0 when the press talk switch 12 is not in operation (OFF) and state 1 when it is in operation (ON). Therefore, when the voice recognition circuit 13 wants to perform voice recognition, the press talk switch 12 is turned on and turned off after the voice input is completed, whereby the voice recognition process based on the corresponding state instruction signal is performed. In units of the speech recognition circuit 13
The voice recognition process is started. Therefore, when a continuous word input is performed by voice, the voice recognition circuit 13 does not analyze whether the input is finished because of OFF (state 0) which means the end of the voice word input. It will be finished. In other words, in the voice recognition circuit 13, when the voice is recognized, the clear start timing and the end timing are known, and the voice recognition range can be easily discriminated by software. On the other hand, it becomes unnecessary to perform unnecessary voice recognition processing. Further, since the handset 10 is not controlled, noise does not occur when switching the voice input (when disconnecting the voice).

Here, the voice output signal from the voice synthesis circuit 19 is sent to the handset of the handset 10.
By performing voice input / output via the handset 10 having the same shape as the handset of the telephone, it is possible to reduce the resistance and discomfort felt when talking to the machine, and when using in a noisy place. It also prevents malfunctions. In addition, by not sending the response sound to the speaker 20 of the television receiver or the stereo set and sending it only to the earpiece of the handset 10, the response sound can be shielded from the co-viewers, and the recording reservation can be made. Setting operations and VTR operations can be performed without inconvenience the joint viewers.

FIG. 2 shows a flowchart of main operations in the apparatus of this embodiment. In FIG. 2, when the operator (user) calls up an animation character by performing a voice input or another input operation in step S1 in an initial state such as after powering on the device,
On the screen SC of the CRT display device 30, for example, FIG.
The animation character AC is displayed as shown in, and the operator enters a standby state for voice input. That is, as shown in FIG. 3, on the screen SC of the CRT display device 30, a balloon for displaying the animation character AC, the utterance content of the animation character AC, and the recognition result of the voice input by the operator in characters. The part SP is displayed in a so-called superimposed manner with the video signal from the VTR 40 as the background image BG. A message (for example, a message such as "What are you doing?" That indicates the standby state) in the speech balloon unit SP is simultaneously formed in the voice synthesis circuit 19 with the same synthesized voice, and the speaker 20 is used. This voice (for example, the voice "What are you doing?") Is pronounced. As a more specific example of the calling process in step S1, the name of the animation character AC (for example, ivy) is called, a call such as "Oi" is input by voice, or the power of the VTR 40 is turned on. When is turned on, first, the animation character AC is displayed together with the message "Yes, it's Ivy." After that, by shifting to the state of FIG. 3, a more natural dialogue atmosphere can be obtained.

In the next step S2, the operator (user) inputs by voice a command and various information for controlling the operation mode designation. This command is the VTR40
If it is a command for operating the traveling system, the process proceeds to step S3 to enter the phase for directly controlling the operation mode of the VTR 40, and if it is a command for recording reservation, the process proceeds to step S4 and recording reservation of the VTR 40. Enter the phase of operation. If it is an instruction to confirm the reservation, step S5
Proceed to and enter the phase of confirming the reserved contents.
In this reservation confirmation phase, the process proceeds to step S6 after step S5, and the reservation is changed or canceled. It is also possible to directly proceed from step S2 to step S6 without passing through step S5.
In the recording reservation and reservation confirmation phases, a deeper level conversation with the animation character AC is performed as described later. These steps S3, S
When the processes of 4, S5 (S6) are completed, the process returns to the voice input standby state of step S2.

Here, in the example of FIG. 3, the animation character AC is displayed on the screen SC of the CRT display device 30, for example, at the lower left. That is, when the animation character AC is displayed in the center of the screen SC, a video (background image B) displayed on the screen is displayed.
G) is visually obstructed, but if it is at the lower left, it is unlikely to obstruct the image on the screen SC. Moreover, the foot of the animation character AC does not float in the air on the lower side of the screen SC, and there is little sense of resistance from the human sensibilities on the lower left side. The balloon portion SP for displaying a message on the screen SC is arranged at the bottom of the screen SC when the character string is written horizontally from left to right, and is arranged vertically when written vertically from top to bottom. For example, it is placed on the right side of the screen SC. This rarely interferes with the image on the screen SC (background image BG).

The motion of the animation character AC is
An animation motion control signal from the control circuit 15 is sent to the animation character generation circuit 16 for control. That is, the animation character generation circuit 16 which receives the above motion control signal from the control circuit 15
Is a speaker for interacting with the operator, for example, the animation character A as shown in FIG.
C outputs a video signal of a moving image in which the mouth is moved according to the voice output of the message, and supplies it to the superimposer 17. The animation signal also includes the display signal of the balloon portion SP that displays the message in text. The super-imposer 17 converts the video of the animation character AC into a video signal in a form of being superimposed on the reproduced video (or television broadcast video) from the VTR 40, so-called superimposing, and sends it to the CRT display device 30. The screen S of the CRT display device 30
The animation character AC projected on C is
It should be an anthropomorphic person with a facial expression and be more familiar. This allows the operator (user) to have a feeling of interacting with the animation character AC (or electronic device).

Next, the voice recognition circuit 13 shown in FIG.
Is for processing the supplied audio signal to recognize the above-mentioned command and various information, and various configurations are conceivable. For example, the configuration shown in FIG. 4 will be described. In FIG. 4, an output signal (input voice signal) from the transmitter 11 of the handset 10 is supplied to the input terminal 21, and the input voice signal is sent to the arithmetic processor 23 via the analog interface 22. Sent.
The analog interface 22 controls the input voice level to a predetermined value according to the control data supplied from the system controller 24, converts the input voice level into a serial digital voice signal, and sends the serial digital voice signal to the arithmetic processor 23. The arithmetic processor 23 forms a voice pattern by, for example, frequency-analyzing the input digital voice signal, and corrects temporal distortion of the voice pattern due to fluctuations in the vocalization rate of human voice (temporal axis normalization). Then, a so-called pattern matching process is performed by comparing the time-axis-normalized voice pattern with a plurality of standard patterns stored in the standard pattern registration memory 25 in advance. The pattern matching process is to calculate the distance between the detected voice pattern and each standard pattern and obtain the shortest standard pattern. The processing result is the control circuit 15 including the CPU or the like via the interface 26. Sent to. Here, in the standard pattern registration memory 25, a plurality of voice patterns such as the above-mentioned command words such as "play" and "stop" and various information words such as "Sunday" and "1 channel" are registered in advance. Accordingly, the arithmetic processor 23 recognizes a word or the like by judging which of the plurality of registered patterns the sound pattern of the input sound signal is (which is the closest). The standard pattern registration memory 25
For registering the pattern in the, the manufacturer may store a plurality of standard voice patterns in advance, or the user may sequentially input a plurality of words by voice before the user starts using the product, and the arithmetic processor 23 may, for example, It is also possible to perform frequency analysis to form the voice patterns and store them.

The output signal from the voice recognition circuit 13 is
It is sent to the control circuit 15, and in this control circuit 15, natural language input processing and inference / interaction processing are performed by a predetermined software program. That is, as shown in FIG. 5, the natural language input processing production system 32 functions in response to the voice input 31 on the user side to form a semantic frame 33 corresponding to, for example, the number of reserved programs. Is sent to the inference / interaction production system 34 to set and control the recording reservation scheduler 35. The natural language input processing production system 32 is a sentence normalization production system PS1,
The phrase segmentation production system PS2, the word extraction production system PS3, and the meaning understanding production system PS4 can be subdivided, and the inference / dialogue production system 34 includes an interpolation inference production system PS5, a habit learning production system PS6, and a dialogue. Processing Production System PS
It can be subdivided into 7. In one meaning frame 33, for example, a plurality of slots such as a day information item, a channel information item, a start time information item, a recording time or an end time information item are provided for recording reservation processing,
Various information input by voice during the reservation process is written in these corresponding item slots.

Next, FIG. 6 is a flow chart for explaining the processing in each step in the flow chart of FIG. 2 in more detail, and an example of performing recording reservation and operation of the traveling system of the VTR 40 is given. . That is, in FIG. 6, in step S11, the operation of calling the animation character AC in step S1 of FIG. 2 is performed. The call in this step S11 is
As described in step S1 of FIG. 2 described above, it is performed in response to voice input of an operator's call command (such as "Ivy" or "Cute") and power-on of the VTR 40. When the calling input of the animation character AC is made in this step S11, the process proceeds to the next step S12, and the animation character AC as shown in FIG. 3 is displayed on the screen SC of the CRT display device 30 (animation character AC startup).

Next, in step S13, it is determined whether the press talk switch 12 is on or off. If it is off (No), the process proceeds to step S14, and if it is on (Yes), the process proceeds to step S15. Step S1
When the operation is proceeded to step 5 and the command from the operator is inputted by voice, first, at step S16, it is judged whether or not the input command is a recording reservation command. If the command is a recording reservation command (Yes), the recording reservation process is performed in step S17, and the process returns to step S13. In the case of No in step S16 (when the command input is not a recording reservation command), the process proceeds to step S18, and it is determined whether or not it is a display command for displaying the contents of the recording reservation already made, and Yes.
In the case of s, in step S19, the process of displaying the previous recording reservation contents on the screen SC of the CRT display device 30 is performed, and then the process returns to step S13, and in the case of No, the process proceeds to step S20. In step S20, it is determined whether the operator's command input is a recording reservation change command. If Yes in this step S20, the recording reservation is changed in step S21 and then the process returns to step S13. If No, the process proceeds to step S22. In step S22, it is determined whether or not it is a recording reservation cancellation command. If Yes, the recording reservation cancellation processing is performed in step S23, and then the process returns to step S13. If No, the process proceeds to step S24. In step S24, it is determined whether the operation command is for the animation character AC,
In the case of Yes, after the operation of the animation character AC is performed in step S25, the process returns to step S13.
As a specific example of the operation of the animation character AC, an operation of causing the animation character AC to be erased from the screen, silenced (silenced), or the like, as described later, can be given. If No in step 24, the process proceeds to step S26. In step S26, it is determined whether or not it is the operation command of the VTR 40. If Yes, the operation process of the VTR 40 is performed, and then the process returns to step S13. If No, it means that the input is the end command and the process is executed. finish.

When the result of the determination in step S13 is No and the process proceeds to step S14, and when the voice command input from the operator (user) is not within a certain time in step S14, for example, the animation character AC is used. The video shows how he has a spare time.
Specifically, for example, at first, the animated character AC yawns and then leans against the edge of the screen,
For example, the user scratches his head, and when there is no voice command input, the animation character AC lays down (sleeps sideways), which is an extra time. This motion processing is hereinafter referred to as leisure animation processing.

FIG. 7 is a flow chart for explaining the details of the reservation process in step S17 of FIG. That is, in FIG. 7, in step S50, a reservation input request display indicating that information input waiting for recording reservation is displayed. As this display, for example, as shown in FIG.
On the screen SC of the CRT display device 30, an animation character AC and a message "Please make a reservation." At this time,
The synthetic voice pronounces "Please make a reservation" and a moving image such as the mouth of the animated character AC moves. In the example of FIG. 8, the character "reservation" indicating the current phase is displayed in the character display window portion PH at the upper right position of the screen SC, for example.

In the state where the display (and voice output) as shown in FIG. 8 is made, the next step S5 in FIG.
In 1, the voice input is performed for various element information for recording reservation of the VTR 40, for example, day of the week, channel, start and end times, and the like. Here, the step S5
In the element information input in 1, it is possible to input a plurality of element information at once and in any order. For this reason, in step S52, the input processing by the natural language input processing production system 32 as shown in FIG. 5 is performed, and the sentence normalization, phrase segmentation, word extraction, and meaning understanding are performed. After that, a plurality of pieces of element information are categorized and written in each item of the meaning frame 33, for example, a day of the week information item, a channel information item, a start time information item, a recording time or an end time information item. After the element information input processing is performed in step S52, it is determined in step S53 whether or not there is insufficient element information.

The lack of element information means that all the item information in the above-mentioned meaning frame is not complete. That is, for example, when four items of a day of week information item, a channel information item, a start time information item, and an end time (or recording time) information item are provided in the meaning frame at the time of recording reservation, If even one of these items lacks information, normal recording reservations cannot be made. Therefore, in step S53, the element information is insufficient (Yes
If it is determined to be s), the question of the insufficient element information is asked in step S66, and the process returns to step S51. The process of complementing the lack information will be described later with reference to FIG.

In step S53 of FIG. 7, there is no shortage (N
If it is determined to be o), the process proceeds to step S54, and the input element information is confirmed. This step S
When confirming the element information 54, a display as shown in FIG. 9 is displayed on the screen SC of the CRT display device 30, for example. As for the display on the screen SC of FIG. 9, as in the case of FIG. 8 above, the text message in the animation character AC and the balloon SP indicates “Is this okay?”.
Is displayed, and the synthesized voice also pronounces "Is this okay?" And also displays that the mouth of the animation character AC moves. Further, the character "reservation" indicating the current phase is displayed in the character display window portion PH of the screen SC. Further, in the confirmation display of FIG. 9, the recording reservation content display window portion P is displayed in the central portion of the screen SC.
R is provided, and in the display window portion PR, for example, recording day data “Wednesday, November 7”, start and end time data “3:00 am to 4:00 am”, and channel data “6 channels” As described above, the data of each item in the meaning frame is displayed.

In the state where the display of FIG. 9 is being performed as described above, in the next step S55 of FIG.
A voice input is made to confirm the reservation at. For example,
A command such as "yes", "return", "no", "change" or voice input of the above element information or the like is performed. Of course, the natural language input processing according to FIG. 5 described above is also performed at this time. In step S56, it is determined whether or not the voice input in step S55 is "yes". If Yes (voice input of "yes") is determined, the process proceeds to step S67 to check whether there is a duplicate recording reservation. To be done. In step S56, No (“Yes”)
If it is determined to be other than), the process proceeds to step S57. In step S57, it is determined whether or not the voice input in step S55 is "return", and if this determination is Yes, it is further determined in step S68 whether or not a question about insufficient information for element information input is queried. . This step S6
If Yes in step 8, the process returns to step S66, and if No, the process returns to step S50. On the other hand, if the determination in step S57 is No, the process proceeds to step S58. In step S58, it is determined whether the voice input in step S55 is element information. If the voice input is element information (Yes), the process returns to step S52, and if it is another voice input that is not element information (No).
Proceeds to step S59. In step S59, it is determined whether or not the voice input in step S55 is "change". If Yes, the process proceeds to step S63, and if No, the process proceeds to step S60. In this step S60, it is selected whether to change or cancel each element information of the recording reservation. Therefore, in step S61, either change / cancellation is input, and the process proceeds to step S62. Step S6
In 2, it is judged again whether or not the voice input is "change". If No, the recording reservation is stopped in step S69. If Yes in step S62, the process proceeds to step S63 to ask a question about the changed content, and then in step S65.
After inputting the element information again in step S52, the process returns to step S52.

FIG. 10 is a flow chart showing details of the element information input processing in step S51 and the element information input processing in step S52 of FIG. FIG. 10 shows a flow of operations focusing on the point that "return" in natural language has a plurality of meanings and different control is required depending on the current state. That is, in FIG. 10, when the element information is input, the semantic analysis of the input element information is performed in step S71, and then the process proceeds to step S72. Step S7
In 2, it is determined whether or not the input voice is "return". If Yes, the process proceeds to step S73, and in step S73, it is determined whether or not the last voice input was "return". Be seen. If Yes in this step S73, the process proceeds to step S75 and the processing for canceling the reservation is performed and the processing is ended. If No is determined in step S73, the process proceeds to step S74, and it is determined in step S74 whether or not there is a question immediately before that.
In the case of No in step S74, the process proceeds to the reservation cancellation in step S75, and in the case of Yes, the process proceeds to step S76. In step S76, the question immediately before is asked, and the process returns to the element information input step of step S51 of FIG. Furthermore, if No in step S72, the process proceeds to step S77. In step S77, it is determined whether or not there is an error, and if the result is Yes, the process proceeds to step S79, and after asking the error item, the above step S5
Return to element information input of 1. No in step S77
If it is determined, the process proceeds to step S78, the writing to the meaning frame is performed, and then the process ends.

In addition to such "return", if a word having a plurality of meanings such as "cancel" or "good" is input by voice, understand the meaning correctly according to the situation. A natural language processing program is built so that it can perform processing, that is, handle polysemy of words.
This can be realized by incorporating the current state into the judgment target, as shown in FIG. 10, so that the operation can be performed with a natural sensation that is normally performed in human language life. In addition, also in the phase of directly operating the traveling system of the VTR 40 in step S3 of FIG. 2 and the reservation confirmation phase of step S5 (and S6), voice input such as “return” and “cancel” Has various meanings, and appropriate measures are taken according to the situation, which will be described later.

Next, FIG. 11 is a flow chart showing the details of the insufficient information processing in FIG. In FIG. 7, presence or absence of various kinds of insufficient information and questions are collectively shown in one step for simplification of illustration, but in FIG. 11, these are expanded according to the four items of the above-mentioned meaning frame, and recording reservation is made. A step corresponding to the presence or absence of element information (hereinafter referred to as HLS data) obtained by learning a habit is also added. That is, in the lack information complementing process of FIG. 11, in the first step S33, it is determined whether or not there is an item of the day of the week information for recording. If yes, the process proceeds to step S36, and if not, the process proceeds to the next step S34. . In step S34, a question as to what day of the week the recording is scheduled is made by displaying the moving image of the animation character AC (displaying the movement of the mouth of the animation character and displaying the message character) and voice, and in step S35, the day of the week information. Is input. In step S36, it is determined whether or not there is an item of channel reservation information, and if yes, the process proceeds to step S39, and if no, the process proceeds to step S37. In step S37, a question is asked as to which channel to set, and in step S38, channel information is input.

Here, in this embodiment, the operator is the VTR.
It has a system (HLS) for learning the habit of reservation when making a recording reservation at 40. That is, in this HLS system, for example, when the information of the day of the week and the channel (or only the day of the week) is input, the recording reservation data (HLS) in which the information of the day of the week and the channel is customarily performed in step S39. Data)), and if it is HLS data, at least the start time and end time of the habitually reserved program are displayed (and audio output) in step S46. Confirm whether to make a custom recording reservation. When such display and confirmation are made, "Yes" or "No" is voice-input in step S47, and the next step S48.
Then, it is judged whether or not to make the above-mentioned habitual recording reservation, and Y
In the case of es (voice input of "yes"), the process proceeds to step S67 of duplication check, and in the case of No, the process proceeds to step S41.

If it is determined in step S39 that the day of the week and channel information is not the customary HLS data, the process proceeds to step S40. In step S40, it is determined whether or not there is information on the start time of the recording reservation,
If yes, the process proceeds to step S43, and if no, the process proceeds to step S41. In step S41, a question is asked as to what time the start time of the recording reservation should be set, and in step S42, information on the start time is input. When this start time information is input, it is determined in step S43 whether there is information about the recording reservation end time or recording time. If yes, the process proceeds to step S49, and if no, the process proceeds to step S44. In step S44, a question is asked as to what time the recording reservation end time is set,
In step S45, the time information of the end time or the information of the recording time is input. In step S49, for example, a display as shown in FIG. 9 (and a voice output such as "Is this okay?") Is made, and it is confirmed whether or not each information for recording reservation input in each of the above steps is correct. Is done.

Next, FIG. 12 is a flow chart for explaining the details of the duplication check processing in step S67 of FIG. In FIG. 12, in step S81, the above-mentioned confirmation of the reservation content is displayed (step S54 in FIG. 7 above).
(Corresponding to steps S55 and S56 in FIG. 7) by voice input to start the duplication confirmation process in step S82. Therefore,
Steps S83 and thereafter correspond to the duplication check process of step S67 in FIG. In step S83 of FIG. 12, it is determined whether or not there is reservation information with overlapping time in the already reserved recording reservation information. If No, the recording reservation is completed in step S92, and the process ends. , Yes, the process proceeds to step S84. In step S84, display of the overlapping recording reservations and selection / change of the overlapping recording reservation contents are selected. In step S85, "change" / "cancel" by voice input
Input either of In step S86, it is determined whether or not the voice input is "change". If No, the recording reservation is stopped and the process ends in step S93, and if Yes, the question of the change content is asked in step S87. do. After step S87, voice input of element information is performed in step S88. If the element information is input in step S88, the semantic analysis process is performed in step S89. At this time, in step S90, it is determined whether or not there is an error in the semantic analysis of voice input. If yes, the error item is queried in step S94, and step S8 is performed.
Return to 8. If it is determined in step S90 that there is none, the process proceeds to step S91, the slot is written in the corresponding information item of the meaning frame, and then the process returns to step S81.

Here, the error processing in steps S90 and S94 and the error processing in steps S77 and S79 in FIG. 10 are, as a general rule, performed every time a voice is input. , There is a grammatical error when a mismatch occurs in the semantic analysis routine (there is no rule) and an input error when there is an error in the element information specification (or there is no error in the specification but it is incorrect in the VTR specifications). Can be mentioned. In the case of the above grammatical error, for example, a message "I didn't understand. Please try again." In the case of the above input error, for example, "I did not understand. What day of the week do you have a reservation?" Or "The start and end times are the same. Please enter the time again." A message for indicating an error item (error item) and requesting re-input is displayed as a moving image (display of the movement of the mouth of the animation character and character display of the message) and voice output, and the input waiting state is set.

Next, FIGS. 13 and 14 are flow charts for explaining the details of the display processing routine in step S19 of FIG. That is, when the display process in step S19 of FIG.
In 101, it is determined whether or not there is information on recording reservation. If not, in step S102, an error message indicating that there is no reservation information to be displayed is displayed as a moving image and voice output, and the process ends. If it is determined in step S101 that the recording reservation content is displayed in step S103, it is determined in step S104 whether or not there is undisplayed reservation information. If it is determined that there is none in step S104, the process proceeds to step S113. If it is determined in step S104, step S1
At 05, it is asked whether to switch the screen of the CRT display device 30. In step S106, a voice input of "yes", "no", "return", a change command, or a cancel command is input. After the input is made in step S106, it is determined in step S108 whether or not the input voice is "yes". This step S1
If Yes is determined in 08, the process proceeds to step S115, and it is further determined whether there is information on the next screen. If Yes is determined in step S115, the process proceeds to step S116, the next screen is displayed, and the process proceeds to step S116.
Returning to step 105, if No is determined in step S115, the process proceeds to step S117, an error message is displayed, and the process returns to step S105. In addition, the above step S1
If the determination in 08 is No, the process proceeds to step S109. In step S109, it is determined whether or not the voice input is "return", and if Yes, the process proceeds to step S118. In step S118, it is determined whether or not there is a previous screen, and if Yes, step S118.
At 119, the previous screen is displayed, the process returns to step S105, and N
If it is o, the process is terminated. Step S109 is N
In the case of o, it progresses to step S110. Step S11
For 0, it is determined whether or not the voice input is a change command. If Yes, the process proceeds to the change process in step S120, and if No, the process proceeds to step S111. Step S
In 111, it is determined whether or not the voice input is a cancel command. If Yes, the process proceeds to the canceling process in step S121, and if No, the process proceeds to step S112. In the step S112, it is judged whether or not the display processing is started from the display instruction, and if No, the step S112.
Return to 110, and if Yes, go to step S113. In step S113, a question is asked as to whether to change or cancel, and in step S114, a voice "yes", "no", "return" or a change command or a cancel command is input.

Further, when the process of step S114 in FIG. 13 is completed, the process proceeds to step S125 shown in FIG. In step S125, it is determined whether or not the voice input in step S114 is a change command.
In the case of es, the change process is performed in step S131 and the post-process is ended. In the case of No, the process proceeds to step S126. In step S126, it is determined whether or not the voice input in step S114 is a cancel command. If Yes, the cancel process is performed in step S132 and the post-processing is terminated, and if No, the process proceeds to step S127. In step S127, it is determined whether or not the voice input in step S114 is "yes". If the determination is No, the process ends, and if the determination is Yes, step S1.
Proceed to 28. In step S128, a question is asked as to whether to change or cancel, and in step S129, a change command, a cancel command, and "return" are input by voice. afterwards,
The process proceeds to step S130, where it is determined whether or not the voice input is "return". If the determination is Yes, the process is terminated, and if No, the above-described step S12.
Return to 5.

FIG. 15 is a flow chart for explaining the details of the change processing routine in step S120 of the flow chart of FIG. That is, in FIG.
In step S140, a recording reservation selection process is performed. Specifically, for example, a list of recording reservation contents is displayed, and a message asking which reservation is to be changed is displayed as a moving image and voice output. Next step S141
Then, if there are, for example, four content display columns in the displayed list, the number of these columns to change the recording reservation information is determined by, for example, "1" to "4". To speak into. In the next step S142, it is determined whether or not there is actually recording-reserved information in the column of the number designated by the voice input. In the case of No in this step S142, an error message is output in step S150 and the process returns to step S140, and in the case of Yes, step S143
Proceed to.

In step S143, a question about the contents of the change regarding the designated recording reservation contents is asked, and in step S144, the voice input of the element information of the contents of the change is performed. In step S145, confirmation processing of the recording reservation content changed by the element information is performed, and step S146 is performed.
The voice inputs "yes", "no" or element information. In step S147, it is determined whether or not the voice input in step S146 is element information. If Yes, the process returns to step S145, and if No, the process proceeds to step S148. This step S148
Then, it is judged whether the above voice input is "yes",
In the case of Yes (voice input of "yes"), step S1
After performing the duplication check process at 51, the process ends. If the determination in step S148 is No, the question of the changed content is asked again in step S149, and then the process returns to step S144.

FIG. 16 is a flow chart for explaining the details of the cancellation processing routine in step S121 of the flow chart of FIG. That is, this FIG.
In step S160, the recording reservation information to be canceled is selected, and in step S161, which recording reservation information among the four recording reservation information of 1 to 4 is to be canceled is input by voice. In step S162, this 1
It is determined whether or not there are recording reservation information of Nos. 4 to 4. If No in step S162, an error message is output in step S168 and the process returns to step S160.
In the case of es, it progresses to step S163. The processing from step S160 to step S162 in FIG. 16 is substantially the same as the processing from step S140 to step S142 in FIG.

Next, in step S163, cancellation confirmation processing is performed, and in step S164, "yes" and "no" are input by voice. In step S165, it is determined whether or not the voice input in step S164 is "yes". If this determination is Yes (voice input is "yes"), the process proceeds to step S166 and the cancellation is completed. To finish. Also, step S165
If the determination is No, the process proceeds to step S167,
The cancellation process is stopped and the process ends.

In the phase of recording reservation or reservation confirmation (and change / cancellation) as described above, FIG.
Steps S54 to S59 of FIG.
105 to S111 and steps S113 to S113 in FIG.
As is clear from step S127 of FIG. 13 and steps S145 to S148 of FIG. 15, inputs other than the requested answer are accepted and understood. That is, in step S54 of FIG. 7, the display (and voice output) as shown in FIG. 9 is performed, and generally, an answer of “yes” or “no” is requested, but directly. By inputting the element information by voice, it is determined as Yes in step S58, and the element information input process S5.
You can move on to 2. Further, by voice inputting "change", it is possible to shift to the change processing. In addition, FIG.
In step S105, the question message as to whether or not the screen is switched is displayed as a moving image and is output by voice. Normally, the answer is “yes” or “no”, but a change command or cancel command is directly input. Then, it is possible to shift to the change processing step S120 and the cancellation processing step S121.

This is required in the usual conversation between humans, in consideration of the fact that the meaning is inferred from the context and a dramatic reply that is not formally succeeded by the other party's premise may be made. The input other than the answer is accepted, and the optimum processing according to the input content is performed. This allows you to skip directly to the “yes” or “no” answer and go directly to the next input step,
The operation procedure can be simplified. In addition, there are a plurality of procedures for moving to a predetermined process. For example, if you want to change the contents of the recording reservation, answer "No" to the question "Is this OK?" At the time of confirmation, and then click "Change" to the question "What would you like to do?" Is the orthodox procedure corresponding to each question, but at the time of the above confirmation,
You can input the command of "change", directly input the changed contents, and return to the information input state immediately before by the command of "return", flexibly responding to various inputs of the user (operator). Can be dealt with. Moreover, it is possible to directly input the element information of the change item, such as "6 channels", to change the information, and not only to simplify the operation procedure,
It is possible to realize an environment or atmosphere of dialogue in natural language in which no obvious thing is mentioned.

FIG. 17 is a flow chart for explaining the details of the operation processing routine of the VTR 40 in step S27 of the flow chart of FIG. That is, in FIG. 17, in step S171, the VTR 40
The current state (eg, which mode the operating mode is in) is checked. In step S172, an operation corresponding to the command corresponding to the operation mode designation control signal is searched from a matrix (command-operation comparison table) for an operation mode described later. A portion of this matrix is shown in Table 1 below.

[Table 1] In the next step S173, it is determined whether or not there is a corresponding operation in this matrix, and if there is no corresponding operation, an error message is output in step S176 and the process ends. If YES in step S173, the flow advances to step S174 to execute the operation corresponding to this command in step S174. Then, step S17
In step 5, a message indicating that the process is executed or a message indicating that the process is executed is issued, and the process ends.

Here, Table 1 shows an example of which operation mode the current operation mode is to be changed to when there is an instruction by the voice input from the current operation mode state. That is, in Table 1, for example, when "power" is input by a voice command, if the current operation mode is, for example, the power off state, the power on operation mode is controlled, and so on. In case of none, stop, play, etc., it is controlled to power off. Further, for example, when “play” is input by the voice command, if the current operation mode is power off, stop, etc., the operation mode is controlled to be reproduction. Further, for example, when the voice input is “fast”, if the current operation mode is, for example, stop, fast-forward, or cue, it is controlled to fast-forward, and if it is playback or pause, it is controlled to cue. If it is a rewind, the rewind is controlled as it is, and if it is a review, the rewind is controlled. Furthermore, for example, in the case of voice input "return", if the current operation mode is, for example, stop or fast forward, rewinding is performed, if it is a pause, review is controlled, and if it is a cue, reverse playback is performed. If it is a rewind, it is controlled to fast forward, and if it is a review, it is controlled to playback. In this way, even for ambiguous voice inputs such as “early” and “return”, the control operation having the correct meaning is selected according to the operation mode at that time.

Next, FIG. 18 is a flow chart for explaining the details of the leisure animation processing routine in step S14 of the flow chart of FIG. In FIG. 18, in step S181, it is determined whether or not it is the top level. This top level is VTR4
It is a level of conversation in which the basic operation of 0 is (directly) performed and the reservation / display phase is entered.
Yes in this step S181 (at the top level)
If it is determined that the top-level leisure animation image is selected in step S186, then step S18 is performed.
Go to 3. Further, if No in step S181, the video of the leisure animation for the processing mode for each phase is selected in step S182. These top-level or processing mode leisure animation images can be preset in various ways. Specifically, for example, as described above, the animation character AC yawns or leans against the edge of the screen. , A motion of scratching the head, and further, a motion of allowing the animation character AC to lie down (sleeping sideways) and to do extra motion. In step S183, a process of selecting one of these leisure animation image patterns is performed by, for example, generating a random number, and then the process proceeds to step S184. In step S184, it is determined whether or not the waiting time when there is no voice input has elapsed, and if No, this determination is repeated, and if Yes, the process proceeds to step S185. In step S185, the image of the leisure animation of the selected pattern is displayed on the screen SC of the CRT display device 30 as a moving image. That is, the animation character AC is caused to perform a motion indicating that it has a spare time.

Further, in order to make the response of the animation character AC more human-like and to give a sense of familiarity, some kind of response (play response) is made to a voice input unrelated to the designation of the operation state of the device. Preferably. To give a concrete example, let's say "Yes, what is it?" To the voice input "Oi", or reply to the voice input "Yoshiyoshi" with "Reply". For example, animation display such as drawing is performed.

By the way, the control circuit 15, the animation character generation circuit 16 and the voice synthesis circuit 19 shown in FIG.
The interface between and is performed using a message packet MP having a predetermined structure. As a specific example of the message packet MP, it is possible to consider one having three elements of a message type indicating the type of message, a message number associated with each message content, and a status. The above message types are, for example, messages without input (for example, "Reserved", "Reservation canceled", etc.)
Instructing to distinguish between input-related messages (see FIGS. 3, 8 and 9 above), display of meaning frame contents, list (list of reservation contents), display of duplicate program list, etc. To do. When the message number is, for example, 8 bits, any of 256 types of messages can be designated.
That is, the control circuit 15 selects an appropriate message type and type (content) according to the recognition data of the input voice, the current state (status), etc., and puts the message type and message number into the message packet MP. And sends it to the animation character generation circuit 16 and the voice synthesis circuit 19. Animation character generation circuit 1
In 6 and the voice synthesis circuit 19, a message corresponding to the message number in the sent message packet MP is displayed as a moving image (display of the movement of the mouth of the animation character and the character display of the message), or voice-synthesized and output. That is why.

It should be noted that in the case of displaying and voice-outputting a message via such a message number, it is necessary to prepare some kinds of messages in advance, and among these plural messages, One message is selected and displayed and output by voice.
Alternatively, the control circuit 15 may be made to compose the message itself, and the composed message may be sent, for example, one character at a time to display a moving image or perform voice synthesis.

Next, the natural language input processing will be further described. For example, at the time of recording reservation, inference of insufficient items such as time information is performed within common sense. For example, if only minutes are specified when entering the start time,
The time is estimated in relation to the current time, and when the hour or minute is input when the end time is input, it is used as the recording time from the start time. Further, when a plurality of element information is input, the relationship between “from” and “up” is taken into consideration, and the voice input such as “up to 30 minutes 9 o'clock” is performed at 8 o'clock 3
It is estimated to be from 0 to 9: 0, or the current time is 8: 1
If it is 0 minutes, it is estimated that it is from 8:30 to 9:30 with respect to the voice input "30 minutes to 1 hour".

The present invention is not limited to the above-described embodiments. For example, a so-called hand microphone may be used as the voice input means in addition to the handset, or a small microphone may be used as a remote control device (so-called remote controller). May be provided. Switch (press talk switch)
May be provided in these hand microphones and remote controls. Further, the electronic device to be controlled is not limited to the VTR, and can be applied to a recording and / or reproducing device for a disc, a control device for various electronic devices such as a digital or analog audio tape recorder, and the like.

[0051]

As described above, in the electronic device control apparatus and method of the present invention, word information representing a command or various kinds of information is extracted from an input signal, and the extracted word information and the learning database are used in accordance with the extracted word information. Controlling the motion of the animated character, controlling the message voice that you want to synthesize, controlling the message signal that instructs the message voice that you want to synthesize, and specifying the input processing action by turning the input state on / off. Thus, for example, even if a part of the order of control commands is omitted or the input order is arbitrary, the action of the animation character and the message voice corresponding thereto can be synthesized, and further various commands from humans can be synthesized. And various information inputs can be handled flexibly, more human input is possible, and anyone can easily control the electronic device. Should be able to, man of a more natural form - has become possible to realize a machine interface.

[Brief description of drawings]

FIG. 1 is a block diagram showing a schematic configuration of a control device of the present embodiment.

FIG. 2 is a flowchart of main operations in the apparatus of this embodiment.

FIG. 3 is a diagram showing a CRT screen on which an animation character and a balloon portion are displayed in an initial state.

FIG. 4 is a block diagram showing a configuration example of a voice recognition circuit.

FIG. 5 is a block diagram for explaining a function of a control circuit.

FIG. 6 is a flowchart showing details of processing in each step in the flowchart of FIG.

7 is a flowchart showing details of a reservation process in the flowchart of FIG.

FIG. 8 is a diagram showing a CRT screen on which a reservation input request is displayed.

FIG. 9 is a view showing a CRT screen on which a display for confirming reservation information is displayed.

10 is a flowchart showing details of element information input and element information input processing in the flowchart of FIG.

11 is a flowchart showing details of insufficient information processing in the flowchart of FIG.

12 is a flowchart showing details of the duplication check process in the flowchart of FIG.

FIG. 13 is a flowchart showing details of display processing in the flowchart of FIG.

FIG. 14 is a continuation of the flowchart of FIG.

FIG. 15 is a flowchart showing details of change processing in the flowchart of FIG.

16 is a flowchart showing details of a cancellation process in the flowchart of FIG.

17 is a flowchart showing details of the VTR operation process in the flowchart of FIG.

FIG. 18 is a flowchart showing details of the leisure animation process in the flowchart of FIG. 6.

[Explanation of symbols]

10 ・ ・ Handset 11 ・ ・ Speaker 12 ... Press talk switch 13 ... Voice recognition circuit 14 ・ ・ Switch state detector 15 ... Control circuit 16 ・ ・ ・ ・ Animated character generation circuit 17: Super imposer 18 ... VTR controller 19 ... Voice synthesis circuit 20 ... ・ Speaker 30 ... CRT display device 40 ... VTR

   ─────────────────────────────────────────────────── ─── Continued front page    (72) Inventor Asako Tamura               6-735 Kita-Shinagawa, Shinagawa-ku, Tokyo               Knee Co., Ltd. (72) Inventor Shiro Omori               6-735 Kita-Shinagawa, Shinagawa-ku, Tokyo               Knee Co., Ltd.                (56) References JP-A-2-86392 (JP, A)                 JP-A-2-83727 (JP, A)                 JP-A-2-141799 (JP, A)

Claims (6)

(57) [Claims] 1. A controller of an electronic device for controlling an electronic device, comprising: switch means for designating an input state; extracting means for extracting word information representing a command or various information from an input signal; and a speaker of a message. An animation character generating means for generating a video signal of an animation character, a voice synthesizing means for synthesizing a voice signal of the message according to a message signal input, and a voice output for outputting the voice signal from the voice synthesizing means as voice. Means, at least a motion control signal for controlling the motion of the animation character of the animation character generating means based on the word information extracted by the extraction means and the information accumulated in the habit learning database , and the voice synthesis. A message that indicates the message voice that you want to synthesize by means of Have a control means for controlling the di signal, said control means, the on / off said switch means
The input state is designated and controlled, and the switch means is also provided.
When the above-mentioned animation character is turned on,
By switching control of the animation character's movement,
The electronic device control is applied to the word information extracted by the extraction means.
It is necessary to judge whether or not the necessary information is missing and
If necessary information is not available
Check whether the information contains the necessary information,
If the learning database contains the necessary information,
The message above to confirm the necessary information by voice output
A control device for electronic equipment, which is characterized by controlling signals . 2. An electronic device control method for controlling an electronic device, comprising: a switch process for designating an input state; an extraction process for extracting word information representing a command or various information from an input signal; and a speaker of a message. An animation character generating step for generating a video signal of an animation character, a voice synthesizing step for synthesizing a voice signal of the message in response to a message signal input, and a voice output for outputting the voice signal from the voice synthesizing step as voice. Step, and at least a motion control signal for controlling the motion of the animation character in the animation character generating step based on the word information extracted in the extraction step and the information accumulated in the habit learning database , and the voice. A message that indicates the message voice that you want to synthesize in the synthesis process Have a control step of controlling the over-di signal, in the above control process, the on / off the switch step
The above input state is specified and controlled, and the above switch
Depending on the turning on in the process, the animation character above
Words extracted by the above extraction process by controlling the operation switching
Information lacks the information necessary to control the above electronic devices
If the necessary information above is missing,
Does the habit learning database include the necessary information?
It is judged whether or not the necessary information is stored in the habit learning database.
If information is included, output the necessary information by voice and confirm
A method of controlling an electronic device, comprising: controlling the message signal so that 3. The control means is a switch of the switch means.
In the meantime, the above animation character has a lot of time to spare.
The control device for an electronic device according to claim 1, wherein processing for expressing the state of being displayed is performed . 4. The switch means specifies a voice input state of an acoustic-electrical conversion element which inputs the command or various information by voice and outputs it as an electric voice signal, and the switch means inputs voice. A voice recognition means for recognizing the command or various information by signal processing the input voice signal from the acoustic-electrical conversion element in which the state is designated is provided, and the extraction means is based on the output signal from the voice recognition means. The control device for an electronic device according to claim 1, wherein the word information is extracted. 5. The control step is the same as the switch step.
In the meantime, the above animation character has a lot of time to spare.
The method for controlling an electronic device according to claim 2, wherein a process of expressing the state of being displayed by a video is performed . 6. In the switch step, the voice input state of an acoustic-electrical conversion element that inputs the command or various information by voice and outputs as an electric voice signal is specified, and the voice input is performed by the switch step. A voice recognition step of recognizing the command or various information by signal processing the input voice signal from the acoustic-electrical conversion element in which the state is designated is provided, and in the extraction step, from the output signal from the voice recognition step. The electronic device control method according to claim 2, wherein the word information is extracted.