GB2387747A

GB2387747A - Inputting speech to a hand held device with multi modal user interfaces via a stylus including a directional microphone

Info

Publication number: GB2387747A
Application number: GB0208899A
Authority: GB
Inventors: James Alexander Rex
Original assignee: Motorola Inc
Current assignee: Motorola Solutions Inc
Priority date: 2002-04-19
Filing date: 2002-04-19
Publication date: 2003-10-22
Anticipated expiration: 2022-04-19
Also published as: WO2003090062A1; GB2387747B; AU2003210299A1; GB0208899D0

Abstract

A communication or computing device comprises a main body 150 and a stylus 110 for interfacing with the main body, wherein the stylus 110 includes a microphone. The microphone is a directional microphone 111, 112, 113 for receiving sounds 101 from a user of the stylus 110, to provide audio input to the device. For a small device operable via a stylus at arm's length from the user, the invention provides higher quality speech input and improved dual (tactile and speech input) modes of operation over existing built-in microphones. The benefits primarily result from the use of a directional microphone positioned within the stylus. The directional microphone may take the form of a microphone array.

Description

- 1 MI CROPHONE ARRANGEMENT

Field of the Invention

This invention relates to a microphone for inputting speech to a handheld device with multi-modal user interfaces. The invention is applicable to, but not limited to, a directional microphone mechanism for use in 10 a stylus to input speech to a hand-held device.

Background of the Invention

In the field of this invention it is known that many new

15 hand-held communication/computing devices have multimodal interfaces, combining video and audio outputs with gestural, tactile and voice inputs. The user is typically provided with the ability to switch rapidly and easily between different modes of input or output, using 20 whichever interface is most convenient for his/her current task.

As communication devices continue to develop, it will often be convenient to provide a voice input 25 simultaneously with tactile input and/or whilst viewing video output. For example, video telephony requires voice input whilst viewing video output, as does voice browsing of WOW pages. It is also convenient to enter text data into multiple form fields of a communication or

30 computing device using a combination of voice and tactile inputs. In this manner, a number of fields or each field

- 2 on a display may be selected by touching the display.

Then text can be entered onto the field of the display by

performing voice-to-text conversion, on speech enunciated into a device's microphone.

In order to view the built-in video display of a hand-

held device, or use its built-in touch pad/screen, the user needs to hold the device at "arm's length". This, however, positions a microphone built into the device 10 relatively far from the user's mouth, resulting in poor quality speech reception.

A standard microphone held at arm's length from a speaker's mouth would receive high levels of noise, 15 relative to sound coming directly from the mouth, in many of the environments where such devices are used (e.g. vehicles, public spaces). Even in quiet environments, arm's-length microphones tend to provide poor quality < - speech, as they receive relatively high levels of speech 20 reverberation.

Strong noise and speech reverberation are annoying to a human listener, and are particularly detrimental to an automatic speech recogniser. To enable voice-input 25 applications on multimodal hand-held devices, it is essential to provide some means of clear speech reception. A common way of ensuring clear speech reception is to 30 place a microphone close to the speaker's mouth. When the main device cannot be held close to the mouth, a

separate microphone can be used, connected to the main unit by a wired or wireless link.

Body-worn microphones (e.g. in headsets) sit close to the 5 mouth. Such microphones are already widely used for hands-free telephony, and may be connected to multimodal hand-held devices. However, many users would rather not carry or wear such microphones, especially if they are using voice input only occasionally.

Hand-held microphones are an alternative, which may be held as close to the mouth as needed. Tactile input to multimodal devices is often provided via a stylus. Hence, to prevent users having two extra hand-held units, it has 15 been recognized as being convenient to integrate the microphone into the stylus.

A stylus microphone can easily be stored in/on the main unit, and can quickly be detached for use. It is 20 particularly convenient if it has a wireless audio link to the main unit.

Previously published descriptions of stylus-mounted

microphones, such as those in patents EP0622724 and 25 W09525326, have made no reference to directionality in their microphones. As mentioned above, prior art

microphones will often provide poor-quality voice reception when held at arm's length from the speaker's mouth.

- 4 However, the inventor of the present invention has recognized that the use of a stylus microphone for tactile input has the disadvantage that the microphone needs to be operable at arm's length from the user's 5 mouth, whilst the stylus is being operated for tactile input. However, for a clear voice input, the microphone would instead need to be held much closer to the speaker's mouth. It is therefore inconvenient to move the stylus microphone back and forth, especially when 10 switching rapidly between voice and tactile inputs (e.g. when selecting fields and entering text into them).

A need therefore exists for an improved microphone in a stylus arrangement wherein the abovementioned 15 disadvantages may be alleviated.

Statement of Invention

O -A In accordance with a first aspect of the present 20 invention, there is provided a communication or computing device, as claimed in claim l.

In accordance with a second aspect of the present invention, there is provided an array microphone in a 25 stylus, as claimed in claim 14.

In accordance with a third aspect of the invention, there is provided a method in accordance with claim 15.

30 Further aspects are as claimed in the dependent claims.

Brief Description of the Drawings

Embodiments of the present invention will now be 5 described, by way of example only, with reference to the accompanying drawings, in which: FIG. l illustrates a generic microphone array arrangement in a stylus, in accordance with a preferred embodiment of 10 the present invention; and FIG. 2 illustrates a stylus microphone array arrangement as envisaged in use, in accordance with the preferred embodiment of the present invention.

Description of Preferred Embodiments

The inventor of the present invention has recognised ; that, instead of placing a stylus microphone closer to 20 the speaker's mouth, a directional microphone can be positioned within the stylus. The directional microphone may be arranged with the axis of the directional microphone substantially parallel with the axis of the stylus. This allows the microphone to pick up the 25 speaker's voice much more strongly than sounds coming from elsewhere.

Pressure-gradient directional microphones (e.g. cardioid, hypercardioid) are small enough to fit inside typical 30 multimodal hand-held devices. However, the inventor of the present invention believes that such microphones are

currently not directional enough for clear speech reception when they are 30cm or further from the mouth, particularly in an acoustically harsh environment.

5 Therefore, for optimum results, the inventor of the present invention further proposes using an array microphone to provide sufficient directionality, thereby exceeding the directionality available from a single directionally-configured microphone.

An array microphone is substantially different to a single directional microphone, as it employs multiple microphones, and the sound signals received by these microphones are processed and combined so as to reduce 15 any signal components from unwanted sources.

However, the multiple microphones, positioned in an array -,; form, must be spaced particular distances apart in order . to perform correctly. The inventor of the present 20 invention has recognized that many array microphone layouts are too large to fit into a typical hand-held device. One type of array, that the inventor has recognized could 25 provide a good performance for this type of application, uses microphones spread along a line about 5cm to 10cm in length. The array needs to be directed towards the speaker's mouth. Standard hand-held device housings are not particularly deep, in the direction towards the 30 user's head. Hence, although such an array could provide a good performance, it is envisaged that this type of

array would not be appropriate to fit inside typical hand-held housings.

The inventor doeshowever envisage that improved single 5 directionallyconfigured microphones may become available in future. These may deliver superior performance to today's microphone arrays, so may in future provide all the advantages of the invention.

10 A further problem with directional microphones, or indeed microphone arrays, is that their performance is reduced when they are not correctly aligned with the speaker's mouth. When a microphone is concealed within a standard-

shaped device housing, the user is not aware of its 15 alignment, if there is any, and may often hold the device in a sub-optimal orientation. When a microphone is housed in a hand-held stylus, the user is more aware of its orientation, and finds it easier to adjust the .... orientation when necessary.

FIG. 1 illustrates a generic example of an array microphone arrangement 100 in a stylus, in accordance with the preferred embodiment of the present invention.

The stylus microphone has a body 110 that may be used as 25 a stylus or pen and which contains two or more microphones 111-113 in an array. The microphones receive sounds coming directly from the mouth of the stylus user 101, as well as other sounds 102.

30 The preferred embodiment of the present invention includes an array processor 120 that processes and

combines multiple microphone output signals to generate a single array output signal 121. The direction of maximal sensitivity 130 of the microphone array is parallel to the long axis of the stylus body 110. Provided that the 5 user's mouth is located close to this direction of maximal sensitivity 130, the array processor 120 will reproduce the speech sounds received directly from the user 101. Concurrently, the array processor 120 substantially suppresses any sounds received from 10 elsewhere 102. The array microphone configuration thus provides a higher quality speech signal than any of its individual microphones in isolation.

The array processor 120 may be housed in the stylus body 15 110, or elsewhere. In the former case, the array output signal 121 is transmitted out of the stylus body. In the latter case, the microphone output signals are transmitted from the stylus body to the array processor 120. Preferably, signals transmitted to or from the 20 stylus body 110 are carried on a link that does not employ a physical connection such as a cable, as a cable tends to hinder manipulation of the stylus. The particular design of this configuration will be influenced by the need for the stylus to be light and 25 easily manipulated by the user. For example, a wireless, infra-red or ultrasonic link may be selected.

The preferred embodiment also includes a multimodal device 150, separate to the stylus body 110, which can 30 simultaneously receive both the array output signal 121 and any positional data 140 from the stylus. The

positional data 140 could, for example, be a men-based' input, i.e. the position is determined by the location on the multimodal device 150 where the stylus makes contact with it. The positional data 140 could also be 'pointer 5 based' input, i.e. determined by the stylus' location or orientation relative to the multimodal device 150.

To provide any kind of positional data 140, the stylus needs to be close to the multimodal device 150. The 10 device also needs to be somewhat distant from the user's head, typically at arm's length, so that the user can clearly see the stylus' position relative to the device, whilst manipulating the stylus. As a result, by providing a stylus microphone array 111, 112, 113, the 15 stylus can be positioned distal to the user's mouth, and yet still provide good speech reception.

Although the preferred embodiment of the present invention is shown having three microphones, it is within 20 the contemplation of the invention that any number of microphones in excess of one, in an array configuration within a stylus, can benefit from the inventive concepts herein described.

25 Referring now to FIG. 2, the stylus arrangement 200 of the preferred embodiment of the present invention is shown in operation. The array microphones 211-213 together with array processor 220 are housed in a stylus 210. The stylus in this configuration is an accessory to 30 a multimodal hand-held computing/communication device 250, which has a touch-sensitive display screen 251. The

- 10 array output signal is preferably transmitted to the multimodal device 250 by a wireless data link 225.

The stylus in such a configuration is preferably small 5 enough to be stowed in, or on, or attached adjacent to, the multimodal device 250. In operation, a stylus user places the stylus tip 219 anywhere on the touchscreen display 251, whilst orienting the stylus 210 so that its direction of maximal sensitivity 230 points towards 10 his/her mouth 200. Advantageously, this enables simultaneous tactile and spoken input to the device 250.

It is preferable for the array's direction of maximal sensitivity to be clearly apparent when looking at the 15 stylus, to help the user orient the stylus for optimal speech reception. The obvious example is when the stylus has a long straight body and the microphone array's a- direction of maximal sensitivity is arranged to be parallel to the stylus body. However, it is within the 20 contemplation of the invention that the direction of maximal sensitivity need not be parallel to the axis of the stylus, and other ways of indicating correct orientation may be used. For example, the stylus could . incorporate a light source or brightlycoloured mark that 25 is only visible to the user when the stylus is correctly oriented with respect to the user.

It is envisaged that, in order to allow hands-free use of the stylus array microphone, the stylus could be mounted 30 on its host device, projecting out towards the user's mouth. Preferably, in such a configuration, the host

device could have a mount for this purpose, such as a well in its front surface. Preferably, the stylus would be mounted removably, and therefore could be mounted by the user temporarily, in the mount.

The inventor of the present invention envisages that new hand-held communication/computing devices with multimodal interfaces may be used for many different applications that involve voice input. For some of these 10 applications, a stylus microphone is convenient. For others, such as voice telephony, a head-mounted microphone may be more convenient. Thus, an alternative application, using a stylus microphone as described above, provides the means of operating the microphone in 15 either hand- held stylus or head-mounted modes.

The "head-mountable stylus microphone" can simultaneously provide tactile/positional input and audio input to the host device. The microphone body is not physically 20 connected to the host device, but communication is provided between them via a wireless, infra red or ultrasonic link. To allow head mounting, it is envisaged that the microphone body is shaped to attach over a user's ear, or elsewhere on the users head.

25 Alternatively, the user may wear a separate head-worn unit, to which the microphone body could be coupled.

Notably, such a head-worn unit could also function as an earpiece (loudspeaker) for audio output from the host device. This arrangement can be considered to be a

- 12 headset with a detachable boom microphone, which is useable as a stylus.

Thus, the above application, providing a microphone 5 stylus that is capable of operating as, or with, a microphone headset, does not need to use an array-based microphone, though an array-based microphone would give better performance than a single directional microphone.

10 It will be understood that the array microphone in a stylus, as described above, provides at least the following advantages: (i) For a small device operable at an arm's length from the user, the inventive concepts hereinbefore described 15 provide higher quality speech input than a built-in microphone, due to the use of a microphone array.

., (ii) Various forms of body-worn microphones could provide high quality speech input to a small device operable at an arm's length from the user. However, for occasional 20 speech input, the inventive concepts are more conveniently provided as a microphone array in a hand-

held device.

(iii) A hand-held microphone array can be used further from the mouth than a single hand-held microphone, thanks 25 to the speech enhancement provided by a microphone array configuration. Furthermore, in combination with its stylus shape, the hand-held device allows the stylus array microphone to provide simultaneous voice and tactile/positional inputs to its multimodal host device.

(iv) A stylus array microphone can assist a user to improve the quality of his/her speech input, as it is easier for the user to align the array microphone correctly, guided, for example, by the shape of the S stylus or some other visual indicator on the stylus. It is also easier for the user to bring the stylus array microphone closer to his/her mouth, if higher quality speech reception becomes necessary (e.g. owing to an increase in background noise).

In summary, a communication or computing device has been

described that incorporates a stylus for interfacing with the device. The stylus includes a directional microphone for receiving sounds from a user of the stylus, to 15 provide audio input to the device. A microphone array in a stylus, adapted for use in the above communication or computing device, has also been described.

. Generally, the inventive concepts contained herein are 20 equally applicable to any suitable communication or computing unit. Whilst specific, and preferred, implementations of the present invention are described above, it is clear that one skilled in the art could readily apply variations and modifications of such 25 inventive concepts.

Thus, an improved microphone in a stylus arrangement has been described wherein the abovementioned disadvantages associated with prior art arrangements have been

30 substantially alleviated.

Claims

- 14 Claims

1. A communication or computing device (110, 150, 5 210, 250) comprising a main body (150, 250) and a stylus (110, 210) for interfacing with the main body wherein the stylus (110) includes a microphone, the communication or computing device characterized in that the microphone is a directional microphone (111, 112, 113) for receiving 10 sounds (101) from a user of the stylus (110), to provide audio input to the main body.

2. The communication or computing device according to Claim 1, wherein the directional microphone includes two 15 or more microphones (111-113) in a microphone array configuration.

3. The communication or computing device according to Claim 1 or Claim 2, wherein the directional microphone is 20 housed in a stylus body (110).

4. The communication or computing device according to Claim 2 or Claim 3, further characterized by the stylus including an array processor (120), operably coupled to 25 the microphone array configuration, to process and combine microphone array output signals into a single output signal (121).

5. The communication or computing device according to any preceding Claim wherein the communication or computing device is a multimodal device (150) , operation of the device being controlled by positioning or moving S the stylus (110) in a particular fashion relative to the main body of the device.

6. The communication or computing device according to Claim 5, wherein the multimodal device (150) is capable 10 of accepting both a directional microphone output (121) and positional data (140) from the stylus (110).

7. The communication or computing device according to any preceding Claim, further characterized by the stylus 15 including a data communication interface such that the stylus' directional microphone is capable of facilitating speech input to a number of devices operable with the -;. data communication interface, in addition to the communication or computing device.

8. The communication or computing device according to Claim 7, wherein the data communication interface facilitates transmission of signals to, or reception of signals from, the communication or computing device via a 25 wireless, infrared or ultrasonic link.

9. The communication or computing device according to any preceding Claim, further characterized by the directional microphone being configured to be 30 substantially parallel to an axis of the stylus for improved audio signal reception.

- 16

10. The communication or computing device according to any preceding Claim, wherein the directional microphone is operably coupled to a signal reception indication 5 function to assist the communication or computing device user to position the stylus for improved speech reception.

11. The communication or computing device according to 10 any preceding Claim, wherein the communication or computing device includes a stylus mount for projecting the stylus substantially towards a user's mouth when the stylus is located in or on the mount.

15

12. The communication or computing device according to any preceding Claim, further characterized by means for mounting the stylus on or adjacent to a user's head, such - that the stylus' directional microphone is directed substantially towards the user's mouth.

13. The communication or computing device according to Claim 12, wherein the means for mounting the stylus on or adjacent to a user's head includes an earpiece loudspeaker, which feeds audio output from the 25 communication or computing device to the user's ear.

14. A microphone array in a stylus adapted for use in the communication or computing device according to any preceding Claim.

15. A method for inputting speech to a communication or computing device (110, 150, 210, 250), comprising a main body (150, 250) and a stylus (110, 150) for interfacing with the main body, wherein the stylus (110) 5 includes a microphone, the method characterized by the microphone being a directional microphone (111, 112, 113), for receiving sounds (101) from the user of the stylus (110), to provide audio input to the main body (150).

16. A method according to Claim 15, wherein the provision of a directional microphone includes two or more microphones (111-113) in a microphone array configuration.

17. A microphone array in a stylus substantially as hereinbefore described with reference to, and/or as illustrated by, FIG. 1 or FIG. 2 of the accompanying drawings.