JPH06110486A - Multimedia device with speech input means - Google Patents

Abstract

PURPOSE:To improve the usability by easily and surely performing a series of operations for speech input by a user in cooperation with various media. CONSTITUTION:When the user stands right in front of the display part 2 of this device, a sensor 51 provided nearby the display screen of the display part 2 among sensors 51-54 and a hook switch 55 constituting an approach detection part 5 turns ON. Then a control part 7 displays and vocalizes a message indicating the receiver of a handset type speech input part 1 is applied to the ear by using the display part 2 and a 1st speech output part 3 provided outside the speech input part 1. Once the application of the receiver to the ear is detected from the states of the sensors 52-54 and hook switch 55 provided at respective parts of the speech input part 1, the control part 7 displays and vocalizes a message requesting, for example, vocalization for ordering by using the display part 2 and a 2nd speech output part 4 incorporated in the speech input part 1.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a multimedia device having voice input means for inputting voice and capable of inputting / outputting a plurality of media.

[0002]

2. Description of the Related Art Recently, in order to improve the operability of voice input,
Handset type voice input devices have been developed. An example of this type of voice input device is shown in Japanese Patent Application No. 63-274692. This Japanese Patent Application Sho 63-27469
The voice input device described in No. 2 determines the timing of accepting voice input by detecting the state of the input device and the operation of the user by the approach detection means (sensor) provided in the grip of the input device. Thus, the ambient noise is prevented from being input.

[0003]

However, although the above-mentioned conventional handset type voice input device can prevent the input of ambient noise and the like, the user must always hold the voice input device and input the voice. There was a problem that it was not easy to use because I did not know when it was good or when to put the voice input device.

The present invention has been made in view of the above circumstances, and an object thereof is to provide a user-friendly multi-user system that allows a user to easily and reliably perform a series of operations for voice input by cooperating with various media. Providing a media device.

[0005]

According to the multimedia device of the present invention, a user uses a handset type voice input means for inputting a voice uttered by the user and the presence and voice input means of the user. A detection means for detecting various actions of the user at the time, and a display message according to the action of the user is displayed on the display means based on the detection content of this detection means, and a voice message according to the action of the user is displayed. And a control means for outputting from the audio output means.

According to the present invention, the voice output means is located near the user's ear when the user uses the voice output means and the first voice output means provided outside the housing of the voice input means. It is also characterized in that it is constituted by the second voice output means incorporated in the housing of the input means, and the output destination of the voice message can be switched.

[0007]

In the above structure, the content of the display message displayed on the display means or the voice is displayed according to the presence or absence of the user detected by the detection means and the operating state of the user when the user exists. The content of the voice message output from the output means is changed by the control means.
In this way, the content of the display message and the content of the voice message are switched according to the operation of the user, so that the user appropriately holds the handset type voice input means according to the display message or the voice message which is switched depending on the operation state of the user. It is possible to surely and easily perform operations such as utterance and confirmation of the display screen.

Further, in the configuration using the two types of voice output means, the first voice output means and the second voice output means, the voice message is output according to the operation state of the user detected by the detection means. The destination is switched. Therefore, when the user approaches the apparatus, a message is output by voice from the first voice output means provided outside the housing of the voice input means, and after the user holds the voice input means, By outputting the message by voice from the second voice output means built in the housing of the input means so as to be located near the user's ear, the voice message is prevented from being input as noise by the voice input means. It is possible to ensure that necessary information reaches the user while preventing others from being asked.

[0009]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to the drawings by taking as an example the case of being applied to a multimedia device used for an ordering machine of a hamburger shop. FIG. 1 is a block diagram schematically showing the configuration of the multimedia device according to the embodiment.

The multimedia device shown in FIG. 1 includes a voice input unit 1, a display unit 2, a first voice output unit 3, a second voice output unit 4, an approach detection unit 5, a voice recognition unit 6 and a control unit 7. Composed of.

The voice input unit 1 is used to input a voice uttered by a user. The display unit 2 is used for displaying menus, messages and the like, and is a CRT display, a liquid crystal display or the like.

The first voice output unit 3 is a speaker (external speaker) used for outputting a message or the like by voice when the user approaches the apparatus, and is provided separately from the housing of the voice input unit 1. To be The second voice output unit is a speaker (internal speaker) used for outputting a message or the like by voice according to the operation content of the user when the user holds (the casing of) the voice input unit 1. The audio input unit 1 is built in the housing.

The approach detection unit 5 is for detecting the presence of the user and various actions of the user when the user uses the voice input unit 1, and optical sensors 51, 52, 5 are provided.
3, 54 and a hook switch 55. The sensor 51 is provided near the display screen of the display unit 2 and is used to detect that the user stands in front of the display unit 2. The sensors 52 to 54 are provided in the housing of the main body portion (input unit 11) of the voice input unit 1 as described later with reference to FIG. Further, the hook switch 55, as will be described later with reference to FIG.
It is provided on a support (12) on which 1) is placed.

The voice recognition unit 6 controls the recognition of the voice input by the voice input unit 1. The control unit 7 causes the display unit 2 to display a display message according to the operation of the user based on the content detected by the approach detection unit 5, and outputs a voice message according to the operation of the user to the first voice output unit 3 or the second voice output unit 3. It is output from the voice output unit 4. Further, the control unit 7 controls the approach detection unit 5
The timing at which the voice recognition unit 6 can start recognition is determined based on the content detected by. The control unit 7 includes a display unit 2
Together with this, it is provided in the housing of the apparatus body. The voice input unit 1 includes a main body portion of the voice input unit 1 shown in the external view of FIG. 2 and the input unit 11 placed in FIG.
And a support base 12 shown in the external view.

As shown in FIG. 2, the housing structure of the input unit 11 is formed in a handset type of a telephone, and includes a grip portion 111 and ears formed at both ends of the grip portion 111. It is configured to have a contact portion 112 and a mouth contact portion 113.

The grip portion 111 is a portion for the user of the multimedia device shown in FIG. 1 to hold the input portion 11.
A sensor 52 is provided on the rear surface at a position slightly closer to the ear rest 112, and a sensor 53 is also provided at a position closer to the mouth rest 113. The grip 111
Has an appropriate length so that when the user holds the input section 11 and brings his / her ear into contact with the earpiece 112, the mouthpiece 113 is located near the mouth of the user.

The sensors 52 and 53 are for detecting that an object or a person has come into contact with or approached, for example, when the user grips the grip 111 with his / her hand. As shown in FIG. 4, the sensors 52 and 53 use the light from the light emitting unit 131 as an object or a human body (here, a hand or the like).
The structure is such that the presence of an object or the like is detected by being reflected by and received by the light receiving unit 132. This sensor 52, 5
The distance of the object or the like that can be detected in 3 can be arbitrarily adjusted, and is usually 2 to 3 cm.

The ear pad 112 has a second voice output shown in FIG. 1 for converting an electric signal from the control unit 7 in FIG. 1 into a sound to generate a synthesized voice (a voice message consisting of the voice). The unit (speaker) 4 is built in. This earpiece 11
A sensor 54 is provided at a substantially central portion of 2. The sensor 54 is for detecting that the ear pad 112 is in contact with or near an object such as a user's ear. The sensor 54 includes the sensors 52,
An optical sensor having the same structure as 53 is used. Also for the sensor 51, an optical sensor having the same structure as the sensors 52 and 53 is used. However, the detection distance of the sensor 51 is several tens of cm.

The mouthpiece unit 113 has a built-in microphone (not shown) at the center of the input unit 11 as an electroacoustic transducer for converting the voice uttered by the user into an electric signal. The electric signal converted by this microphone is supplied to the voice recognition unit 6 shown in FIG. Further, the detection signals of the sensors 52 to 54 are supplied to the control unit 7 shown in FIG. 1 via the same code 117.

As shown in FIG. 3, the support base 12 has the ear pad 112 and the mouth pad 113 facing downward, and the input section 1 is provided.
It has recesses 121 and 122 into which the ear rest 112 and the mouth rest 113 fit when the device 1 is placed. The input portion 11 is provided in the recess 121 into which the ear pad 112 fits.
A hook switch 55 is provided for detecting that the is placed on the support 12. This hook switch 55
Turns off when the input unit 11 is placed on the support base 12,
It is turned on when the input unit 11 is picked up. Instead of the hook switch 55, an optical sensor having the same structure as the sensors 51 to 54 can be used. in this case,
The detection distance is 2-3 cm. Next, the operation of the embodiment of the present invention will be described.

First, the sensor 51 constituting the approach detection unit 5
The on / off state signals, which are detection signals of 54 to 54 and the hook switch 55, are guided to the control unit 7. Based on the on / off state signals (on / off state) of the sensors 51 to 54 and the hook switch 55, the control unit 7 determines the presence / absence of a user and the content of the user's operation (action) on the input unit 11. , Can be determined as shown in FIG.

That is, if the sensor 51 (provided in the vicinity of the display screen of the display unit 2) is on, the control unit 7 indicates that the user is present in front of (the display unit of) the present multimedia device (1 ), And if the sensor 51 is off, it determines the state (2) in which the user does not exist in front of (the display unit of) the present multimedia device.

Further, in the control unit 7, at least one of the sensors 52 and 53 provided on the rear surface of the input unit 11 near the ear rest 112 and the position near the mouth rest 113 is turned on, and the control unit 7 is provided on the ear rest 112. If the sensor 54 is turned on and the hook switch 55 is also turned on, the state (3) in which the input unit 11 is correctly applied to the user's ear is determined.

Next, the sensors 52 and 53 are both turned off,
If the sensor 54 is on and the hook switch 55 is off, the control unit 7 determines the state (4) in which the input unit 11 is correctly placed on the support base 12.

Next, if at least one of the sensors 52 and 53 is on, the sensor 54 is off, and the hook switch 55 is on, the control section 7 causes the surface (upper surface) of the input section 11 shown in FIG. The state (5) in which the input portion 11 is picked up but not placed on the ear (8) is determined.

Next, the sensors 52 and 53 are both turned off,
When the sensor 54 is off and the hook switch 55 is on, the control unit 7 turns down the surface (one side surface) of the input unit 11 shown in FIG. 2B or the opposite surface (the other side surface). The state (6) placed on a table or the like is determined.

Next, if any of the sensors 52 and 53 is off, the sensor 54 is on, and the hook switch 55 is on, the control section 7 changes the surface (lower surface) of the input section 11 shown in FIG. It is determined whether it is placed down on a table or the like (7), or when the input unit 11 is put on the ear but is held poorly (9).

When the user stands in front of the display unit 2 of the present multimedia device, the display unit 2 of the sensors 51 to 54 and the hook switch 55 constituting the approach detection unit 5 is displayed.
The sensor 51 provided near the display screen is turned on. Then, the control unit 7 determines the state (1) shown in FIG.
The process according to the flowchart shown in is started.

First, the control unit 7 displays, on the display unit 2, a guide screen including a message such as "Please put the handset on your ear," as shown in FIG. 7 (step S).
1). At the same time, the control unit 7 causes the first voice output unit (external speaker) 3 to output a voice message, for example, "Please bring your handset." At this time, the input unit 11 (receiver) of the voice input unit 1 is in the state (4) shown in FIG.
53 is off, the sensor 54 is on, and the hook switch 55 is off.

The user standing in front of the display unit 2 of the present apparatus first grasps the grip 111 of the input unit 11 and supports the input unit 11 in accordance with the display contents (see FIG. 7) and the voice output contents. Pick up from the table 12. Thus, when the user sets the input unit 11 to the state (8) shown in FIG. 5, at least one of the sensors 52 and 53 is turned on, the sensor 54 is turned off, and the hook switch 55 is turned on. When the user picks up the input unit 11 as described above, the user puts the ear pad 112 of the input unit 11 on his / her ear.

As described above, in this embodiment, the user simply stands in front of (the display unit 2 of) the multimedia device having the configuration shown in FIG. 1 to display the guide screen as shown in FIG. , The voice message “Please bring the handset” is output from the first voice output unit 3 which is an external speaker, and the user accordingly picks up the input unit 11 of the voice input unit 1 from the support base 12. The earpiece 112
Can be applied to your ears.

Now, after the user picks up the input section 11,
When the earpiece 112 is correctly applied to the ear, the sensor 5
At least one of 2, 53 is ON, hook switch 55
The sensor 54 is turned on while the signal is kept on.

Then, the control section 7 determines that the input section 11 is in the state (3) shown in FIG. 5, that is, the user correctly holds the input section 11 and correctly puts the ear pad 112 on his / her ear. It is determined that the state has been reached (steps S2, S
3). In this case, the control unit 7 determines that the order by voice using the input unit 11 can be correctly performed,
For example, as shown in FIG. 8, a menu screen including a message "Please say your desired product name." Is displayed on the display unit 2 (step S4). At the same time, the control unit 7
The ear pad 11 provided on the input unit 11 (of the voice input unit 1
The second voice output unit (internal speaker) 4 incorporated in 2) outputs a voice message such as "Please speak." Further, when the control unit 7 determines that it is possible to place an order by voice from the user, the control unit 7 accepts the voice input from (the input unit 11 of) the voice input unit 1 thereafter and sets it as a recognition target. The voice recognition unit 6 is notified.

While looking at the menu screen shown in FIG. 8, the user picks up the input unit 11 as described above and pushes the ear pad 11
With 2 applied to the ear, and thus with the mouthpiece 113 approaching the mouth, the desired product name may be uttered in accordance with the above-mentioned voice message "Please speak." For example, if the user says "hamburger""1",
This voice is converted into an electric signal by a microphone built in the ear pad 112 of the input unit 11, sent to the voice recognition unit 6, and recognized by the recognition unit 6. In addition, the user continues,
For example, if "grape juice""one" is uttered, the voice recognition unit 6 similarly recognizes it.

At this time, even if the user utters for ordering without waiting for the output of the voice message "Please speak", the voice message will still be input.
Second audio output unit 4 built in the ear pad 112 of the
Since it is output from, there is no possibility that it will be picked up by a microphone built in the mouthpiece part 113 of the input part 11 and become noise. Of course, if only the voice (electrical signal) input from the input unit 11 (the microphone built in the mouthpiece unit 113) is accepted after the output of the voice message “Please speak.” There is no problem even if the voice message is output from the first voice output unit 3, which is an external speaker. However, considering that there are users who can not wait until the output of the voice message is completed, as described above, the input unit 11 becomes the state (3) shown in FIG. 5 in steps S2 and S3. It is more convenient to accept the input voice after the time when it is determined and output the voice message "Please speak." From the second voice output unit 4, which is an internal speaker. By the way, the voice recognition result in the voice recognition unit 6, that is, the voice recognition result of the order contents by the voice from the user is sent to the control unit 7 every time the recognition is performed. The control unit 7 receives the recognition result of the voice recognition unit 6 and clearly indicates the recognized order contents on the menu screen displayed on the display unit 2 as shown in FIG. 9 (step S5).
S6). As this explicit method (emphasized display method), various methods such as reverse display, blink display, display with different colors, and display surrounded by a frame can be applied.

After that, no utterance occurs even after a predetermined period of time elapses, and the input unit 11 (handset) is held by the support base 12.
If it is not placed in (steps S7 to S1)
0), the control unit 7 indicates that the order is completed, for example, "I have one hamburger and one grape juice right now. Please place the handset if you like."
Voice message, that is, a message requesting confirmation of the order result by the user and a message for prompting the operation of ending the voice input, the earpiece unit 11 provided in the input unit 11 (of the voice input unit 1
The second audio output unit 4 built in 2) is output (step S11).

When the user places the input unit 11 (receiver) on the support 12, the sensors 52 and 53 are both off, the sensor 54 is on, and the hook switch 55 is off. Then, the control unit 7 determines that the input unit 11 is in the state (4) shown in FIG. 5, and recognizes that the order clearly indicated on the menu screen has been decided (step S1).
2, S13).

On the other hand, if the user utters "No" or "correction" when the order contents are incorrect, the uttered contents are recognized by the voice recognition unit 6 and passed to the control unit 7. At the stage, a message such as "Please reorder" is displayed on the display unit 2 and the second voice output unit 4 outputs the voice (steps S14 and S15). As a result, it becomes possible to place an order (reload) by voice input again. In addition, even if the user puts the handset (input unit 11) on the support base 12 before the voice output of the message requesting confirmation of the user's order result and the voice input termination operation, that state Is determined (step S8,
In S9), the order is determined in the same manner as above. The control unit 7
When recognizing that the order details have been decided, for example, as shown in FIG.
As shown in, the contents of the order and the amount of the order are displayed on the display unit 2 (step S16).

In the above description, steps S2 and S
After the state (3) shown in FIG. 5 is discriminated in step 3 until the message “Please put the handset on” is output in step S11, the user uses the earpiece 1 of the input section 11.
It is assumed that 12 is correctly applied to the ear. For this reason, the voice message is output from the second voice output unit 4 built in the input unit 11.

However, in this voice output system, if the user releases his / her ear from the ear pad 112 of the input unit 11, the voice message output from the second voice output unit 4 will be the user's ear. Does not reach Therefore, when outputting the voice message, the state of the input unit 11 is checked by the sensors 52 and 53, the sensor 54, and the hook switch 55, and the sensor 5
At least one of 2, 53 is on, the sensor 54 is off,
The state (8) shown in FIG. 5 in which the hook switch 55 is turned on.
In the case of, that is, when the input unit 11 is picked up, but the ear pad 112 is not put on the ear (8), a voice message is also output from the external first voice output unit 3. Then, the user may listen.

When a message is displayed or voice output from the apparatus, if the user does not handle the input section 11 as expected, for example, for voice output of "Please bring the handset", If the input unit 11 does not reach the state (3) shown in FIG. 5, a message indicating that the input unit 11 is mishandled and a guidance message for prompting correct handling are displayed or output by voice. Good.

The above description has been given of the case where the present invention is applied to a multimedia device used for a hamburger shop ordering machine, but the present invention is not limited to this, and the present invention can be applied to all multimedia devices provided with a voice input section. .

[0043]

As described above, according to the multimedia device of the present invention, the presence of the user and various actions of the user when the user uses the handset type voice input means are detected by the detecting means. Since the display screen and audio output contents are switched based on this detection content, it is possible to dynamically switch to an appropriate display screen and audio output content that always follows the user's operation, and the user can Depending on the display screen and the voice output contents that are switched depending on the state, it is possible to reliably and easily carry out operations such as appropriate holding of the handset type voice input means, utterance, and confirmation of the display screen.

Further, according to the present invention, the voice input means includes a voice output means (second voice output means) built in and a voice output means (first voice output means) provided outside the voice input means. By configuring to switch the audio output destination according to the operating state of the user detected by the detection means,
It is possible to prevent the voice from being input as noise into the voice inputting means and to prevent it from being heard by others, while surely delivering the necessary information to the user.

[Brief description of drawings]

FIG. 1 is a block diagram showing a schematic configuration of a multimedia device according to an embodiment of the present invention.

FIG. 2 is a main body portion of the voice input unit 1 in FIG. 1 (input unit 1
The figure which shows the external appearance of 1).

FIG. 3 is a main part of the voice input unit 1 (input unit 1 in FIG. 1).
The figure which shows the external appearance of the part (support stand 12) which pairs with 1).

4 is an external view showing the configuration of sensors 52 and 53 in FIG.

5 shows the relationship between the presence / absence of a user and the operation (operation) state of the user with respect to the input unit 11, and the states of the sensors 51 to 54 and the hook switch 55 which form the approach detection unit 5 in FIG. FIG.

FIG. 6 is a flowchart for explaining a series of operations by the control unit 7 in the apparatus when it is detected that the user stands in front of the apparatus in FIG.

7 is a diagram showing an example of a display screen displayed when it is detected that the user stands in front of the device of FIG.

FIG. 8 is a diagram showing an example of a display screen displayed when the user picks up the input unit 11 of the voice input unit 1 and puts it on the ear.

FIG. 9 is a diagram showing an example of a display screen displayed when completion of an order from a user is detected.

FIG. 10 is a diagram showing an example of a display screen displayed when it is detected that the input unit 11 is placed on the support base 12 after the order is completed.

[Explanation of symbols]

1 ... voice input unit, 2 ... display unit, 3 ... first voice output unit,
4 ... 2nd audio | voice output part, 5 ... approach detection part, 6 ... voice recognition part, 7 ... control part, 11 ... input part, 12 ... support base 51-
54 ... Sensor, 55 ... Hook switch, 111 ... Grip, 112 ... Ear rest, 113 ... Mouth rest.

Claims (6)

[Claims] 1. A voice input unit for inputting a voice uttered by a user, a display unit used for displaying a message and the like, a presence of the user, and a use when the user uses the voice input unit. A detection unit for detecting various actions of the user, and a control unit for displaying a display message according to the action of the user on the display unit based on the detection content of the detection unit. A multimedia device characterized in that message contents are switched according to the above. 2. A voice input means for inputting a voice uttered by a user, a voice output means used for voice output of a message and the like, a presence of the user and a case where the user uses the voice input means. A detection means for detecting various actions of the user, and a control means for causing the voice output means to output a voice message according to the action of the user based on the detection content of the detection means. A multimedia device characterized in that the content of a message is switched according to the operation of a person. 3. A voice input means for inputting a voice uttered by a user, a display means used for displaying a message and the like, a voice output means used for voice output of a message and the like, and presence and presence of the user. A detection unit for detecting various actions of the user when the user uses the voice input unit, and a display message according to the action of the user is displayed on the display unit based on the detection content of the detection unit. And a control means for outputting a voice message according to the user's operation from the voice output means, and the display message content and the voice message content are switched according to the user's operation. apparatus. 4. A handset type voice input means for inputting a voice uttered by a user, and a first voice output means provided outside the housing of the voice input means and used for voice output of a message or the like. And, when the user uses the voice input unit, the voice input unit is built in the housing of the input unit so as to be located near the user's ear.
Second voice output means used for voice output of a message, etc., detection means for detecting the presence of the user and various actions of the user when the user uses the voice input means, and this detection means And a control unit for outputting a voice message according to the user's action from the first voice output unit or the second voice output unit based on the detected content of the message, and switching the message content according to the user's action. In addition, the multimedia device is characterized in that the output destination of the message is switched. 5. A handset type voice input means for inputting a voice uttered by a user, a display means used for displaying a message etc., and a message etc. provided outside the housing of the voice input means. A first voice output unit used for voice output, and a built-in housing of the input unit so that the voice input unit is located near a user's ear when the user uses the voice input unit,
Second voice output means used for voice output of a message, etc., detection means for detecting the presence of the user and various actions of the user when the user uses the voice input means, and this detection means Based on the detected content of the above, a control for displaying a display message according to the operation of the user on the display means and outputting a voice message according to the operation of the user from the first voice output means or the second voice output means. A multimedia device comprising: means for switching display message contents and voice message contents according to a user's operation, and switching an output destination of the voice message. 6. A voice recognition unit for recognizing a voice input by the voice input unit is further provided, wherein the voice recognition unit starts preparation for recognition in accordance with a user's operation detected by the detection unit. The multimedia device according to any one of claims 1 to 5, which is characterized.