GB2377805A - Localisation of a person in a conveyance - Google Patents

Localisation of a person in a conveyance Download PDF

Info

Publication number
GB2377805A
GB2377805A GB0116772A GB0116772A GB2377805A GB 2377805 A GB2377805 A GB 2377805A GB 0116772 A GB0116772 A GB 0116772A GB 0116772 A GB0116772 A GB 0116772A GB 2377805 A GB2377805 A GB 2377805A
Authority
GB
United Kingdom
Prior art keywords
mirror
input device
user
vehicle
adjustable
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
GB0116772A
Other versions
GB0116772D0 (en
GB2377805B (en
Inventor
Robert William Series
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
20 20 Speech Ltd
Original Assignee
20 20 Speech Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 20 20 Speech Ltd filed Critical 20 20 Speech Ltd
Priority to GB0116772A priority Critical patent/GB2377805B/en
Publication of GB0116772D0 publication Critical patent/GB0116772D0/en
Publication of GB2377805A publication Critical patent/GB2377805A/en
Application granted granted Critical
Publication of GB2377805B publication Critical patent/GB2377805B/en
Anticipated expiration legal-status Critical
Expired - Fee Related legal-status Critical Current

Links

Classifications

    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60RVEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R21/00Arrangements or fittings on vehicles for protecting or preventing injuries to occupants or pedestrians in case of accidents or other traffic risks
    • B60R21/01Electrical circuits for triggering passive safety arrangements, e.g. airbags, safety belt tighteners, in case of vehicle accidents or impending vehicle accidents
    • B60R21/015Electrical circuits for triggering passive safety arrangements, e.g. airbags, safety belt tighteners, in case of vehicle accidents or impending vehicle accidents including means for detecting the presence or position of passengers, passenger seats or child seats, and the related safety parameters therefor, e.g. speed or timing of airbag inflation in relation to occupant position or seat belt use
    • B60R21/01512Passenger detection systems
    • B60R21/0153Passenger detection systems using field detection presence sensors
    • B60R21/01538Passenger detection systems using field detection presence sensors for image processing, e.g. cameras or sensor arrays
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60RVEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R21/00Arrangements or fittings on vehicles for protecting or preventing injuries to occupants or pedestrians in case of accidents or other traffic risks
    • B60R21/01Electrical circuits for triggering passive safety arrangements, e.g. airbags, safety belt tighteners, in case of vehicle accidents or impending vehicle accidents
    • B60R21/015Electrical circuits for triggering passive safety arrangements, e.g. airbags, safety belt tighteners, in case of vehicle accidents or impending vehicle accidents including means for detecting the presence or position of passengers, passenger seats or child seats, and the related safety parameters therefor, e.g. speed or timing of airbag inflation in relation to occupant position or seat belt use
    • B60R21/01512Passenger detection systems
    • B60R21/01552Passenger detection systems detecting position of specific human body parts, e.g. face, eyes or hands
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00Speech recognition
    • G10L15/24Speech recognition using non-acoustical features
    • G10L15/25Speech recognition using non-acoustical features using position of the lips, movement of the lips or face analysis
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60RVEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R21/00Arrangements or fittings on vehicles for protecting or preventing injuries to occupants or pedestrians in case of accidents or other traffic risks
    • B60R21/01Electrical circuits for triggering passive safety arrangements, e.g. airbags, safety belt tighteners, in case of vehicle accidents or impending vehicle accidents
    • B60R2021/01204Actuation parameters of safety arrangents
    • B60R2021/01252Devices other than bags
    • B60R2021/01265Seat belts
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00Processing of the speech or voice signal to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/0208Noise filtering
    • G10L21/0216Noise filtering characterised by the method used for estimating noise
    • G10L2021/02161Number of inputs available containing the signal or the noise to be suppressed
    • G10L2021/02166Microphone arrays; Beamforming

Abstract

An input system for providing an input to an automatic speech recogniser 160 in a conveyance, e.g. an automobile 100, has a directional input device 170 and an adjustable mirror 110, the position of adjustment of the mirror being effective to influence a direction of sensitivity of the input device.

Description

<Desc/Clms Page number 1>
LOCALISATION OF A PERSON IN A CONVEYANCE This invention relates to localisation of a person in a conveyance. In particular, it relates to localisation of a person in a conveyance with the aim of optimising the performance of an automatic speech recognition system.
There are several conditions in which localisation of a user can enhance the ability of an automatic speech recognition system to recognise the speech of the user. For example, it has been found that an automatic speech recognition system that combines analysis of audio speech signals with analysis of a video signal of a speaker's mouth can give an improvement in recogniser performance equivalent to lOdB of noise reduction in an audio-only system. However, such systems can be implemented efficiently and economically only if the user's mouth can be localised within the field of view of a video input device. As a further example, use of a directional microphone can reduce noise as compared with a wide-field microphone. However, this is effective only if the user's
<Desc/Clms Page number 2>
mouth and microphone's beam remain in alignment. US-A- 5687280 discloses a system that can assist a user in achieving such alignment with a hand-held device.
In both of the above cases, it may be possible to use advanced signal processing techniques to locate and track a user's mouth within the field of a microphone or video device. However, such techniques are of limited applicability and invariably require hardware of such processing power that they cannot be implemented in massproduced, low-cost systems. In other cases, is possible to ensure that a user's mouth is always close to a particular location when they are engaging with the system. However, this is not possible in many cases, for example, in the case where a user is an operator of a vehicle.
An aim of this invention is to provide at least some of the enhancements in recogniser performance available from localisation of a user's mouth without resorting to the approaches discussed in the last-preceding paragraph.
The present inventors have realised that there is an aspect of a user's behaviour that is highly dependent upon the position of a user's eyes, and in consequence, the user's mouth, and it is from this realisation that the first aspect of the invention has arisen.
From a first aspect, this invention provides an
<Desc/Clms Page number 3>
input system for providing an input to an automatic speech recogniser in a vehicle, the system comprising a directional input device and an adjustable driving mirror, the position of adjustment of the driving mirror being effective to influence a direction of sensitivity of the input device.
The normal behaviour of a vehicle operator is to adjust a driving mirror so that they get a good rearward view of the road to the rear of the vehicle. Therefore, given an understanding of the geometry of the mirror and the vehicle, an inference can be drawn as to the position of the operator's eyes, and, therefore, the operator's mouth.
The input device may include a directional microphone, such as a beam-steering microphone.
Alternatively or additionally it may include a video input device such as a video camera, which might be a charge coupled device video camera.
In a first application of this invention, the influence of the driving mirror may be directly applied.
For example, the relative position of the mirror and the video camera may be arranged such that the mirror, when adjusted by a user, reflects an image of the user's mouth towards the camera. The mirror therefore serves the function of optically controlling the direction of (at
<Desc/Clms Page number 4>
least some of) the camera's visual field towards the user's mouth.
A system embodying this aspect of the invention may include a processor that extracts a portion, for example the lips, of the image of the user's mouth. Alternatively or in addition, a system embodying this aspect of the invention may include a processor that interprets an output from the video camera and makes an inference as to a position or aspect of the user's mouth. Once the inference is made, such embodiments will typically generate an output that is indicative of the result of the inference. Each inference or a series of inferences made over a period of time may be indicative of one or more speech elements spoken by the user. The inferences may be used to enhance or even replace recognition of the user's speech by recognition of audio signals. Thus, the system acts to read the user's lips.
Most advantageously, the adjustable driving mirror may be (conventionally) provided on the vehicle to perform a task other than its association with this invention. For example, the adjustable driving mirror may be provided on the vehicle as a matter of routine. In the case of a motor land vehicle, the adjustable mirror may be a rear view mirror, or a mirror attached to the rear view mirror (which may, for example, be convex to achieve a broader
<Desc/Clms Page number 5>
field of view for an optical input device).
In certain embodiments, the video camera may be located by the vehicle rear window with the camera's visual field directed towards the adjustable driving mirror. Where the adjustable driving mirror is a rear view mirror, the video camera might be located above the vehicle rear window. During installation, the video camera is positioned so that at least some of its visual field is directed towards the reflection in the rear view mirror of a user's mouth. If a user changes his location in the vehicle, e. g. by adjusting the height of his seat, the user may be expected to adjust the rear view mirror so that he gets a good rearward view of the road to the rear of the vehicle. Such adjustment causes the rear view mirror to direct the reflection of the user's mouth towards the video camera.
Adjusting the direction of the reflection of the user's mouth, as described in the last-preceding paragraph, works especially well where the change in the fore-and-aft position of the user is small in comparison with the distance between the video camera and the mirror.
For example, the system works especially well where the change in the position of the user's mouth in mainly in the vertical direction, as may happen when the height of the user's seat is altered. The size of the error caused
<Desc/Clms Page number 6>
by a change in the fore-and-aft position of the driver may be determined by simple geometry. Where the error is of a size that the reflected image of the user's mouth is out of the video camera's visual field it may be necessary to adjust the direction of the camera's visual field. Thus, according to embodiments of the invention the direction of the video camera's visual field may be manually adjustable in relation to the driving mirror. Alternatively or in addition, the direction of the video camera's visual field in relation to the driving mirror may be remotely adjusted by mechanical means, such as one or more electric motor.
According to certain embodiments, the system may include a sensor for measuring the fore-and-aft position of the user, and the measured position may be used to make an inference as to the size of the error referred to in the last-preceding paragraph. In such embodiments, the system may provide a warning to the user of the need to change the direction of the video camera's visual field. Alternatively or in addition, the system may automatically change the direction of the video camera's visual field, e. g. by means of one or more electric motors.
In another application of this invention, the influence of the driving mirror may be indirectly applied.
For example, the system may include one or more sensors to detect the position of the mirror, and a processing stage
<Desc/Clms Page number 7>
to analyse an output from the (or each) sensor and form an inference as to the location of the user's mouth. Such a system typically further comprises a control stage operative to adjust the direction of sensitivity of the input device in response to the inferred location of the user's mouth.
This application of the invention is particularly advantageous where the input device can be controlled indirectly. For example, it may be applied where the input device is a beam-steering microphone that can be controlled electronically or in software analysis. As a further example, it may be applied to select a restricted region of the field of a comparatively wide-angle video input device to produce an effective directional video input.
A system applied as described in the last two preceding paragraphs may typically be used to form an inference as to the direction of the user's mouth in relation to the driving mirror. For example, the driving mirror may be a rear view mirror and the one (or more) sensors may be associated with the rear view mirror to form an inference as to the direction of the user's mouth in relation to the input device by analysis of the output of the (or each) sensor.
According to an enhancement of the system of the
<Desc/Clms Page number 8>
last-preceding paragraph, the sensor (s) may be used to detect the position of more than one driving mirror. For example, the driving mirrors may be a rear view mirror and a door mounted mirror, and the processing stage may form an inference as to the direction of the user's mouth in relation to each mirror.
In certain embodiments, the inferred directional information for each mirror may be used to form an inference as to the three-dimensional location of the user1 s mouth within the vehicle. The inferred threedimensional location of the user's mouth may be used to control a characteristic of the input device. For example, where the input device is a video input device the inferred three-dimensional location of the user's mouth might be used to adjust focus or a depth of field of the video input device.
From a second aspect, the invention provides a functional system for installation in a vehicle having an input system according to the first aspect of the invention. The functional system most typically includes an automatic speech recogniser.
In particular, the system may be operative to control operation of the input device. This system may be applied to any input device that has indirect control (as described above) of its direction of sensitivity, such as
<Desc/Clms Page number 9>
a beam-steering microphone or a video input device.
In a favoured application of this aspect of the invention, the functional system includes an automatic speech recogniser and is operative to direct an input device, such as a beam-steering microphone or a comparatively wide-angle video input device, to provide an input for the automatic speech recogniser. More specifically, interpretation of the output from the one or more sensors controls the beam direction of the beamsteering microphone, and the extent of the field of view of the video input device. In such embodiments, the system is operative to generate an output that is indicative after suitable processing of the location of the user's mouth. The control stage receives the system output and adjusts the direction of sensitivity of the beam-steering microphone or video input device. This can be beneficial to the quality of signal provided by the microphone or video input device to the automatic speech recognition device.
From a third aspect, this invention provides a vehicle having a functional system that embodies the second aspect of the invention. For example, the vehicle may be a motor land vehicle such as a car.
From a fourth aspect of the present invention, there is provided a method of providing an input to an automatic
<Desc/Clms Page number 10>
speech recogniser in a vehicle, in which the condition of adjustment of an adjustable driving mirror is determined, the condition of adjustment being effective to influence a direction of sensitivity of an input device.
A method embodying the fourth aspect of the invention may determine the condition of more than one adjustable driving mirror and combine such determinations in influencing the direction of sensitivity of the input device.
It is to be appreciated that the method according to this aspect of the invention may include any one or more of the features described above with reference to the preceding aspects of the invention.
Having arrived at this invention, the inventors realised that it has broader application than has been described above. There are there are further fittings and controls of a vehicle that are adjusted by a user from which an inference could be drawn as to the user's location. Moreover, there may be other systems in the vehicle, the performance of which can be enhanced through use of knowledge of the user's location.
Therefore, from a fifth aspect, the invention provides a sensing system for providing an output indicative of a user's location in a conveyance comprising a sensor associated with apparatus installed on the
<Desc/Clms Page number 11>
vehicle, the sensor providing an output indicative of the state of adjustment of the apparatus, the apparatus being of a type that is adjusted by a user of the conveyance in relation to their position within the conveyance.
Very many vehicles contain suitable apparatus. By detecting their state of adjustment, an inference can be made as to the location of a user that has adjusted the apparatus in order to use it.
For example, the adjustable apparatus may be an adjustable mirror, such as a rear view mirror. In such embodiments, the sensor output may indicate the direction in which the mirror glass has been directed, having been adjusted by the user. It is reasonable to assume that a user will adjust a mirror to provide a good rear view.
This allows an inference to be made to determine the position of the user's eyes (and therefore, their mouth) with reference to the state of adjustment of the mirror and the position of, for example, the rear window of the vehicle. An analogous inference can be drawn in the case where the mirror is an external mirror. The output of a system of this type might, for example, be used to control the beam direction of a directional microphone or the direction in which the output of a display is directed.
The adjustable apparatus might also be an adjustable seat. Such seats, in a vehicle, are typically adjustable
<Desc/Clms Page number 12>
in several modes, for example, for fore-and-aft position, seat back rake, height, and so forth. Each of these modes might present a respective sensor output. In this case, an inference can be made to the position of the user's body. The output of a system such as this might, for example, be used to control deployment of an air bag.
The adjustable apparatus may, for example, be manually adjusted or adjusted (e. g. remotely) by mechanical means, such as one or more electric motor.
It is to be appreciated that the term'sensor', as used herein, is not limited to the definition, typically used in the field of instrumentation, of a device that converts a physical quantity into an electrical quantity.
Rather, is to be interpreted as meaning any arrangement that provides an appropriate output for the particular installation. Thus, the term'sensor', as used herein, might mean, for example, include a device that converts a mechanical stimulus to an electrical response and viceversa, a device that converts a mechanical stimulus to a mechanical response, or a device that converts an electrical stimulus to an electrical response.
The system of the present invention may include one or more of a wide range of sensor types. For example, the sensor may include an electrical sensor, an electromechanical sensor or a mechanical sensor.
<Desc/Clms Page number 13>
Moreover, the system may include sensors that are associated with several such adjustable apparatus within the vehicle. For example, the system may include sensors that are associated with an internal rear-view mirror and one or more external door mirrors of a vehicle.
A system embodying the present invention may be used to control a functional system on the vehicle. Typically, this is done with the aim of optimising the performance of the functional system.
A system embodying the invention may further include a processor that interprets the output from the sensor and makes an inference as to the position of a user within the vehicle. Once the inference is made, such embodiments will typically generate an output that is indicative of the result of the inference.
In this respect, from seventh aspect, the invention provides a functional system for installation in a vehicle having a sensing system according to the fifth aspect of the invention.
In particular, the sensing system may be operative to control operation of a user interface of the functional system. In general, this aspect of the invention may be applied to any system that might benefit from being provided with information relating to the location of a person in a vehicle. The functional system might include
<Desc/Clms Page number 14>
one or more of an electro-acoustic transducer, such as a microphone, a display such as a head-up or holographic display, an air bag deployment system or the like.
In a particularly favoured application of this aspect of the invention, the functional system includes an automatic speech recogniser, and the sensing system is operative to direct a microphone to provide an input for the automatic speech recogniser. More specifically, the microphone may have a steerable beam, the direction of which is controlled by the interpretation of the output of the sensing system. A particular application of such embodiments is to control systems in a motor land vehicle (e. g. a car). In such an application, the adjustable apparatus might be a rear view mirror of the vehicle (which is adjusted by the vehicle driver to optimise his view through a rear window). In such embodiments, the sensing system is operative to generate an output that is indicative (usually, after suitable processing) of the likely position of the vehicle driver's head. The functional system receives the output of the sensing system and controls the microphone so that it is directed to receive sounds originating from the vicinity of the driver's head. This can be very beneficial to the quality of a signal that the microphone supplies to the automatic speech recogniser.
<Desc/Clms Page number 15>
Alternatively or in addition, the functional system may include a display. In such cases, the output of the sensing system might be used to improve a vehicle driver's view of the display by controlling the orientation of the display in relation to the vehicle driver.
Alternatively or in addition, the functional system may include an air-bag restraint. In such embodiments, the sensor output might be used to improve upon the capability of one or more air bags to arrest movement of the vehicle driver by controlling the (e. g. direction and/or position of) deployment of the air bags in the event of the vehicle being involved in an accident.
Most advantageously, the adjustable apparatus may be (conventionally) provided on the vehicle to perform a task other than its association with this invention. For example, it may be a control, accessory, component etc. that would be provided on the vehicle as a matter of routine. In the case of a motor land vehicle, the apparatus may be a mirror (e. g. an internal rear-view mirror or a door-mounted mirror).
Vehicles with which embodiments of the present invention can be used include conveyances such as land vehicles (e. g. a car), rail vehicles, water-borne vessels or aircraft.
From an eighth aspect, this invention provides a
<Desc/Clms Page number 16>
vehicle having a functional system that embodies the seventh aspect of the invention. For example, the invention may provide a vehicle (e. g. a motor land vehicle such as a car) or an aircraft or water craft, in which the functional system is controlled by an automatic speech recogniser, and in which the sensing system provides an output that is used to control the beam of a beam-steering microphone that provides an input for the automatic speech recogniser. The functional system is most typically used to control aspects of operation of the vehicle (including its fittings, accessories, ancillary components etc).
From a further aspect of the present invention, there is provided a method of providing an output indicative of a location of a person in a vehicle in which the condition of adjustment of adjustable apparatus within the vehicle is determined, and, having regard to the condition of adjustment, an inference is made as to the position of a person for whom the apparatus has been adjusted.
Naturally, the advantage of this aspect of the invention is achieved only if the adjustable apparatus is properly adjusted. Therefore, this aspect of the invention may further include a step of issuing a reminder or an instruction to the user to properly adjust the apparatus.
A method embodying the invention may determine the
<Desc/Clms Page number 17>
condition of adjustment of more than one adjustable apparatus, and combine such determinations in making its inference.
It is to be appreciated that the method of providing an output indicative of a location of a person in a vehicle may include any one or more of the features described above with reference to the apparatus for providing an output indicative of a location of a person in a vehicle.
A specific embodiment of the present invention will now be described, by way of example, and with reference to the accompanying drawings in which: Figure 1 is a side view of a car being a first embodiment of the present invention; Figure 2 is a flow chart representation of a first method embodying the present invention; Figure 3 is a side view of a car being a second embodiment of the present invention; Figure 4 is a flow chart representation of a second method embodying the present invention; and Figure 5 is a side view of a car being a third embodiment of the present invention.
With reference to Figure 1, a first system embodying the present invention is embodied in a vehicle in the form of a car 100. The car 100 has a rear view mirror 110
<Desc/Clms Page number 18>
(which constitutes an adjustable driving mirror) that can be adjusted by a driver 120 of the car. A video camera 130 (which constitutes a directional input device) is installed behind the driver 120 and above the car rear window and senses an image of the user's mouth reflected by the rear view mirror towards the camera's visual field.
The car also has a processor 140 that can interpret the output from the video camera 130, by extracting a portion of sensed image of the user's mouth comprising the lips and making an inference as to a position or aspect of the user's lips. The position or aspect of the user's lips is indicative of one or more speech elements spoken by the user. The processor 140 can generate an output that is indicative of the result of the inference.
The car 100 of this embodiment includes audio equipment 150 that can be controlled by voice commands. Voice commands are provided to the audio equipment 150 by an automatic speech recogniser 160. A microphone 170 provides an input to the automatic speech recogniser 160.
The output of the processor 140, which is indicative of one or more speech elements spoken by the user, is used to enhance the quality of the speech detected by the microphone 170.
The components of the embodiment can be permanently installed on the car 100, for instance during manufacture.
<Desc/Clms Page number 19>
Alternatively, the video camera 130 and the processor 140 can be provided as a kit for installation, perhaps temporarily, on the car 100. The kit might also include the audio equipment 150, the automatic speech recogniser 160 and the microphone 170, where these components are not already installed. Installing the kit involves attaching the video camera to the car behind the driver, e. g. above the rear view window, and adjusting the rear view mirror so that it provides a good rear view. The position of the video camera is then adjusted so that the image of the user's mouth reflected in the rear view mirror falls within the camera's field of view, as indicated in Figure 1 by the broken line between the person (represented by the broken lines) 120 and the mirror 110 and the solid line between the mirror 110 and the video camera 130. It is a notable advantage that the mirror serves the function of optically controlling the direction of the camera's visual field towards the user's mouth. A further notable advantage is that the video camera is used in conjunction with a component (the rear view mirror in the present embodiment) that is provided on the car as a matter of routine to perform a task other than putting this embodiment of the invention into effect.
According to a mode of operation and with reference to both figures 1 and 2, a driver 120 adjusts his rear
<Desc/Clms Page number 20>
view mirror 110 to provide a good rear view. For example, the driver 120 may need to adjust the rear view mirror as a result of lowering the height of his seat, as illustrated in Figure 1 by the representation of the driver 125 drawn in solid lines. The adjustment of the rear view mirror causes the reflection of the driver's mouth to be directed towards the visual field of the video camera, as indicated in Figure 1 by the solid line between the driver's lips and the mirror 110 and the solid line between the mirror 110 and the video camera 130. Based on the output from the video camera the processor 140 makes an inference as to a position or aspect of the driver's lips 210 (which constitutes determining a condition of adjustment of an adjustable driving mirror), which is indicative of one or more speech elements spoken by the driver. The output of the processor is provided to the automatic speech recogniser 160 to enhance the speech detected by the microphone 170,220 (which constitutes influencing a direction of sensitivity of an input device).
With reference to Figure 3, a second system embodying the present invention is embodied in a conveyance in the form of a car 300. The car 300 has a rear view mirror 310 (which constitutes an adjustable apparatus) that can be adjusted by a driver 320 of the car. The rear view mirror
<Desc/Clms Page number 21>
310 is provided with a transducer 330 (which constitutes a sensor) that can generate a signal indicative of the position of angular adjustment of the rear view mirror 310. The transducer 330 might be an electromechanical sensor such as an inductive displacement sensor. The car also has a processor 340 that can interpret the output from the transducer 330 and make an inference as to the position of the driver 320 within the car 300. The processor 340 can generate an output that is indicative of the result of the inference.
The car 300 of this embodiment includes audio equipment 350 that can be controlled by voice commands (the audio equipment constituting a functional system of the vehicle). Voice commands are provided to the audio equipment 350 by an automatic speech recogniser 360. A beam-steering microphone 370 (which constitutes a user interface of the functional system) provides an input to the automatic speech recogniser 360. The output of the processor 340 is used to control a beam of the beamsteering microphone 370 so that it is directed to receive voice commands.
The components of the embodiment can be permanently installed on the car 300, for instance during manufacture. Alternatively, the transducer 330 and the processor 340 (which constitute a functional system for installation in
<Desc/Clms Page number 22>
the conveyance) can be provided as a kit for installation, perhaps temporarily, on the car 300. The kit might also include the audio equipment 350, the automatic speech recogniser 360 and the beam-steering microphone 370, where these components are not already installed. Installing the kit involves aligning the transducer 330 with the rear view mirror 310. It is a notable advantage that the transducer is associated with a component (the rear view mirror in the present embodiment) that is provided on the car as a matter of routine to perform a task other than putting the present invention into effect.
According to a mode of operation and with reference to both Figures 3 and 4, the driver 320 adjusts the rear view mirror 310 to provide a good rear view. The transducer 330 provides an output that is indicative of the direction in which the rear view mirror glass has been directed 410 (which constitutes determining a condition of adjustment of the apparatus). Based on the output of the transducer and with reference to the position of the rear window of the car 300, the processor 340 makes an inference as to the position of the driver's eyes, and hence his mouth 430 (which constitutes making an inference as to the position of a person). The output of the processor 340, which is indicative of the result of the inference, causes the beam of the beam-steering microphone
<Desc/Clms Page number 23>
370 to be steered so that it is directed to provide for optimal reception of voice commands given by the driver 310.
The mode of operation can include a step of issuing a reminder or an instruction to the user to adjust the rear view mirror 310 to provide a good rear view 440 (which constitutes issuing a reminder to properly adjust the apparatus). Thus, the advantage of the present invention can be fully achieved. The mode of operation can also include determining the condition of adjustment of another adjustable apparatus 420, such as an external mirror, by the same means discussed above in relation to the rear view mirror. The processor 340 then makes an inference as to the position of the driver's eyes based on both the determined condition of the rear view mirror and the determined condition of the external mirror.
With reference to Figure 5, a third system embodying the present invention is embodied in a conveyance in the form of a car 500. The car 500 has an adjustable seat 510 (which constitutes an apparatus) that can be adjusted by a driver 520 of the car. The seat of this embodiment has two modes of adjustment, namely for fore-and-aft position and seat back rake. First and second transducers 530,540 (which constitute first and second sensors) are associated with the adjustable seat 510 and each of the two modes of
<Desc/Clms Page number 24>
adjustment presents a respective transducer output. The first transducer 530, which presents an output indicative of fore-and-aft position, might for instance have the form of one or more switches that are arranged along one of the seat runners. Moving the seat in relation to the seat runner causes the switches to be actuated in turn to generate a signal indicative of fore-and-aft position adjustment. The second transducer 540, which presents an output indicative of seat back rake, might for instance be a rotary displacement transducer.
The car 500 of Figure 5 also has a processor 550 that can interpret the outputs from the first and second transducers 530,540 and make an inference as to the position of the driver's body within the car. In addition, an air-bag restraint 560 is provided in the car 500. The output of the processor 550, which is indicative of the result of the inference, is used to control deployment of the air-bag restraint 560.
According to a mode of operation, the driver 520 adjusts his seat 510 for comfort. The driver might adjust his seat 510 manually. Alternatively, the seat might be adjusted by electric motors that are controlled remotely by the driver, for example by means of touch switches.
The first and second transducers 530,540 provide outputs to the processor 550, thereby allowing the processor 550
<Desc/Clms Page number 25>
to make an inference as to the position of the driver's body within the car 500. In the event of the car being involved in an accident, the output of the processor 550 is used to control the direction and/or position of deployment of the air-bag restraint 560. Thus, the capability of the air-bag to arrest movement of the driver is improved.
It is to be appreciated that the systems, methods and other features described with reference to the embodiments discussed above can be combined in other embodiments of the present invention.

Claims (64)

  1. CLAIMS: 1. An input system for providing an input to an automatic speech recogniser in a conveyance, the system comprising a directional input device and an adjustable mirror, the position of adjustment of the mirror being effective to influence a direction of sensitivity of the input device.
  2. 2. A system as claimed in claim 1, in which the input device includes a directional microphone, such as a beam-steering microphone.
  3. 3. A system as claimed in claim 1 or claim 2, in which the input device includes a video input device such as a video camera, which might be a charge coupled device video camera.
  4. 4. A system as claimed in any preceding claim, in which a relative position of the adjustable driving mirror and the video input device are arranged such that the mirror, when adjusted by a user, reflects an image of the user's mouth towards the camera.
    <Desc/Clms Page number 27>
  5. 5. A system as claimed in any preceding claim, further including a processor for extracting a portion, such as the lips, of the image of a user's mouth.
  6. 6. A system as claimed in any preceding claim, further including a processor for interpreting an output from the video input device and making an inference as to a position or aspect of the user's mouth.
  7. 7. A system as claimed in claim 6, in which the system generates an output that is indicative of the result of the inference, each inference or a series of such inferences made over a period of time being indicative of one or more speech elements spoken by the user.
  8. 8. A system as claimed in any preceding claim, in which the adjustable driving mirror is a rear view mirror.
  9. 9. A system as claimed in any preceding claim, in which the video input device is located by a vehicle rear window with a visual field of the video input device directed towards the adjustable driving mirror.
  10. 10. A system as claimed in any preceding claim, in which
    <Desc/Clms Page number 28>
    a direction of a visual field of the video input device is manually adjustable in relation to the adjustable driving mirror.
  11. 11. A system as claimed in any preceding claim, in which a direction of a visual field of the video input device is remotely adjusted by mechanical means, such as one or more electric motor, in relation to the driving mirror.
  12. 12. A system as claimed in any preceding claim, further including a sensor for measuring a fore-and-aft position of a user.
  13. 13. A system as claimed in claim 12, in which the system provides a warning to the user of a need to change a direction of a visual field of a video input device.
  14. 14. A system as claimed in claim 12 or 13, in which the system automatically changes a direction of a visual field of the video input device, e. g. by means of one or more electric motors.
  15. 15. A system as claimed in any preceding claim, further including one or more sensors to detect the position
    <Desc/Clms Page number 29>
    of the adjustable driving mirror, and a processing stage to analyse an output from the (or each) sensor and form an inference as to the location of a user's mouth.
  16. 16. A system as claimed in claim 15, further comprising a control stage operative to adjust the direction of sensitivity of the input device in response to the inferred location of the user's mouth.
  17. 17. A system as claimed in claim 15 or claim 16, in which the input device is a beam-steering microphone that can be controlled electronically or in software analysis.
  18. 18. A system as claimed in any one of claims 15 to 17, in which the input device is a comparatively wide-angle video input device and the inference is applied to select a restricted region of the field of the comparatively wide-angle video input device to produce an effective directional video input.
  19. 19. A system as claimed in any one of claims 15 to 18, in which the sensor (s) may be used to detect a position of more than one driving mirror.
    <Desc/Clms Page number 30>
  20. 20. A system as claimed in claim 19, in which the more than one driving mirror is a rear view mirror and a door mounted mirror, and the processing stage forms an inference as to a direction of a user's mouth in relation to each mirror.
  21. 21. A system as claimed in claim 20, in which the inferred directional information for each mirror may be used to form an inference as to the three- dimensional location of the user's mouth within the vehicle.
  22. 22. A functional system for installation in a vehicle having an input system according to any one preceding claim.
  23. 23. A functional system as claimed in claim 22, in which the system is operative to control operation of the input device.
  24. 24. A functional system as claimed in claim 22 or claim 23, in which the functional system includes an automatic speech recogniser and is operative to direct an input device, such as a beam-steering
    <Desc/Clms Page number 31>
    microphone or a comparatively wide-angle video input device, to provide an input for the automatic speech recogniser.
  25. 25. A functional system as claimed in claim 24, in which the system is operative to generate an output that is indicative after suitable processing of the location of the user's mouth.
  26. 26. A vehicle having a system according to any preceding claim.
  27. 27. A method of providing an input to an automatic speech recogniser in a vehicle, in which the condition of adjustment of an adjustable driving mirror is determined, the condition of adjustment being effective to influence a direction of sensitivity of an input device.
  28. 28. A method as claimed in claim 27, in which the method determines the condition of more than one adjustable driving mirror and combines such determinations in influencing the direction of sensitivity of the input device.
    <Desc/Clms Page number 32>
  29. 29. A sensing system for providing an output indicative of a user's location in a conveyance comprising a sensor associated with apparatus installed on the conveyance, the sensor providing an output indicative of the state of adjustment of the apparatus, the apparatus being of a type that is adjusted by a user of the conveyance in relation to their position within the conveyance.
  30. 30. A sensing system according to claim 29 in which the adjustable apparatus includes an adjustable mirror.
  31. 31. A sensing system according to claim 30 in which the mirror is a vehicle rear view mirror.
  32. 32. A sensing system according to claim 30 or claim 31 in which the sensor output indicates the direction in which the mirror glass has been directed.
  33. 33. A sensing system according to any one of claims 29 to 32, in which the adjustable apparatus includes an adjustable seat.
  34. 34. A sensing system according to claim 33 in which each of a plurality of modes of adjustment of the seat
    <Desc/Clms Page number 33>
    presents a respective sensor output.
  35. 35. A sensing system according to any one of claims 29 to 34, in which the adjustable apparatus is manually adjusted.
  36. 36. A sensing system according to any one of claims 29 to 35, in which the adjustable apparatus is (remotely) adjusted by mechanical means, such as one or more electric motor.
  37. 37. A sensing system according to any one of claims 29 to 36 that includes one or more of a range of sensor types.
  38. 38. A sensing system according to claim 37, in which the sensor includes one or more of an electrical sensor, an electromechanical sensor or a mechanical sensor.
  39. 39. A sensing system according to any one of claims 29 to 38 comprising a sensor associated with each of several such adjustable apparatus within the vehicle.
  40. 40. A sensing system according to claim 39 that includes sensors that are associated with an internal rear-
    <Desc/Clms Page number 34>
    view mirror and one or more external door mirrors of a vehicle.
  41. 41. A sensing system according to any one of claims 29 to 43 used to control a functional system on the conveyance.
  42. 42. A sensing system according to any one of claims 29 to 41, further including a processor that interprets the output from the sensor and makes an inference as to the position of a user within the conveyance.
  43. 43. A sensing system according to claim 42 in which, once the inference'is made the processor generates an output that is indicative of the result of the inference.
  44. 44. A functional system for installation in a conveyance having a sensing system according to any one of claims 29 to 43.
  45. 45. A functional system according to claim 44 in which the sensing system is operative to control operation of a user interface of the functional system.
    <Desc/Clms Page number 35>
  46. 46. A functional system according to claim 44 or claim 45 that includes one or more of an electro-acoustic transducer, a display such as a head-up or holographic display, or an air bag deployment system.
  47. 47. A functional system according to any one of claims 44 to 46 that includes an automatic speech recogniser.
  48. 48. A functional system according to claim 47 in which the sensing system is operative to direct a microphone to provide an input for the automatic speech recogniser.
  49. 49. A functional system according to claim 48 in which the microphone has a steerable beam, the direction of which is controlled by the interpretation of the output of the sensing system.
  50. 50. A functional system according to claim 49 in which the adjustable apparatus is a rear view mirror of the vehicle.
  51. 51. A functional system according to claim 49 or claim 50 in which the sensing system is operative to generate an output that is indicative (after suitable
    <Desc/Clms Page number 36>
    processing) of the likely position of the vehicle driver's head.
  52. 52. A functional system according to claim 51 in which the output of the sensing system is used to control a microphone so that it is directed to receive sounds originating from the vicinity of a driver's head.
  53. 53. A functional system according to any one of claims 44 to 52 including a display.
  54. 54. A functional system according to any one of claims 44 to 53 including an air-bag restraint.
  55. 55. A system according to any one of claims 29 to 43 in which the adjustable apparatus is (conventionally) provided on the conveyance to perform a task other than its association with this invention.
  56. 56. A system according to claim 55 in which the adjustable apparatus is a control, accessory, component etc. that would be provided on the conveyance as a matter of routine.
  57. 57. A system according to claim 56 in which the
    <Desc/Clms Page number 37>
    adjustable apparatus includes a mirror such as an internal rear-view mirror or a door-mounted mirror.
  58. 58. A conveyance having a system according to any one of claims 29 to 57.
  59. 59. A conveyance according to claim 58 that is a vehicle (e. g. a motor land vehicle such as a car) or an aircraft or water craft, in which operation of the functional system is controlled by an automatic speech recogniser, and in which the sensing system provides an output that is used to control the beam of a beam-steering microphone that provides an input for the automatic speech recogniser.
  60. 60. A vehicle according to claim 59 in which the functional system is used to control aspects of operation of the vehicle, including its fittings, accessories, ancillary components etc.
  61. 61. A method of providing an output indicative of a location of a person in a conveyance in which the condition of adjustment of adjustable apparatus within the conveyance is determined, and, having regard to the condition of adjustment, an inference
    <Desc/Clms Page number 38>
    is made as to the position of a person for whom the apparatus has been adjusted.
  62. 62. A method according to claim 61 including a step of issuing a reminder or an instruction to the user to properly adjust the apparatus.
  63. 63. A method according to claim 61 or claim 62 including determining the condition of adjustment of more than one adjustable apparatus, and combining such determinations in making its inference.
  64. 64. A system substantially as described herein with reference to the accompanying drawings.
GB0116772A 2001-07-10 2001-07-10 Method of and apparatus for providing an input to an automatic speech recogniser in a conveyance Expired - Fee Related GB2377805B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
GB0116772A GB2377805B (en) 2001-07-10 2001-07-10 Method of and apparatus for providing an input to an automatic speech recogniser in a conveyance

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
GB0116772A GB2377805B (en) 2001-07-10 2001-07-10 Method of and apparatus for providing an input to an automatic speech recogniser in a conveyance

Publications (3)

Publication Number Publication Date
GB0116772D0 GB0116772D0 (en) 2001-08-29
GB2377805A true GB2377805A (en) 2003-01-22
GB2377805B GB2377805B (en) 2003-12-17

Family

ID=9918193

Family Applications (1)

Application Number Title Priority Date Filing Date
GB0116772A Expired - Fee Related GB2377805B (en) 2001-07-10 2001-07-10 Method of and apparatus for providing an input to an automatic speech recogniser in a conveyance

Country Status (1)

Country Link
GB (1) GB2377805B (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2312579A1 (en) * 2009-10-15 2011-04-20 Honda Research Institute Europe GmbH Speech from noise separation with reference information
EP2977986A1 (en) * 2014-07-21 2016-01-27 Honeywell International Inc. Audio command adaptive processing system and method
US9609408B2 (en) * 2014-06-03 2017-03-28 GM Global Technology Operations LLC Directional control of a vehicle microphone

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5884254A (en) * 1995-08-02 1999-03-16 Sensimetrics Corporation Method and apparatus for restricting microphone acceptance angle
US6053529A (en) * 1997-12-22 2000-04-25 Ford Global Technologies, Inc. Occupant restraint system with seat position sensor

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5884254A (en) * 1995-08-02 1999-03-16 Sensimetrics Corporation Method and apparatus for restricting microphone acceptance angle
US6053529A (en) * 1997-12-22 2000-04-25 Ford Global Technologies, Inc. Occupant restraint system with seat position sensor

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2312579A1 (en) * 2009-10-15 2011-04-20 Honda Research Institute Europe GmbH Speech from noise separation with reference information
US9609408B2 (en) * 2014-06-03 2017-03-28 GM Global Technology Operations LLC Directional control of a vehicle microphone
EP2977986A1 (en) * 2014-07-21 2016-01-27 Honeywell International Inc. Audio command adaptive processing system and method
US10276180B2 (en) 2014-07-21 2019-04-30 Honeywell International Inc. Audio command adaptive processing system and method

Also Published As

Publication number Publication date
GB0116772D0 (en) 2001-08-29
GB2377805B (en) 2003-12-17

Similar Documents

Publication Publication Date Title
US5450057A (en) Stereophonic warning apparatus
EP1695873B1 (en) Vehicle speech recognition system
EP1908640B1 (en) Voice control of vehicular elements from outside a vehicular cabin
US7467809B2 (en) Vehicular occupant characteristic determination system and method
EP1884421B1 (en) Method and system for processing voice commands in a vehicle enviroment
US6942248B2 (en) Occupant restraint device control system and method
AU2001264210A1 (en) Safety devices for use in motor vehicles
JP2007512182A (en) Automatic visual recognition of vehicle blind spots
US20090153665A1 (en) Lane change control system
EP3531398B1 (en) Rear lateral side warning apparatus and method with learning of driving pattern
JP3446805B2 (en) Voice input device for vehicles
CN113016016A (en) Information presentation control device, information presentation control method, program, and recording medium
US7231051B2 (en) Detection of viewing direction by microphone
GB2377805A (en) Localisation of a person in a conveyance
JP3841329B2 (en) Vehicle display device
JP2001352595A (en) Voice input device
CN112672920A (en) Method for operating a driver assistance system, vehicle, and computer program and data carrier signal
KR100505881B1 (en) System auto controlling the side-mirror in a car
KR100523578B1 (en) System auto controlling the rear-mirror in a car
CN108885865B (en) Vehicle effect sound generating device
KR101571594B1 (en) Apparatus and Method for adjusting the angle of side mirror in vehicle
CN112867440A (en) Occupant monitoring system for a vehicle
CN117641291A (en) Vehicle with internal/external communication system
CN116533232A (en) Control method and device of vehicle-mounted robot and vehicle-mounted robot
JP4259257B2 (en) Vehicle door mirror

Legal Events

Date Code Title Description
PCNP Patent ceased through non-payment of renewal fee

Effective date: 20100710