GB2570629A

GB2570629A - Vehicle controller

Info

Publication number: GB2570629A
Application number: GB1719063.8A
Authority: GB
Inventors: Hasedzic Elvir
Original assignee: Jaguar Land Rover Ltd
Current assignee: Jaguar Land Rover Ltd
Priority date: 2017-11-17
Filing date: 2017-11-17
Publication date: 2019-08-07
Anticipated expiration: 2037-11-17
Also published as: GB2570629B; DE102018218875A1; GB201719063D0

Abstract

A method of controlling operation of a rear view imaging device (15 see fig 1) for a vehicle (43 see fig 8), the rear view imaging device (15) including a mirror or a display screen 71. The method comprises detecting a hand 17 of a vehicle occupant within a volume of space within a vehicle cabin; determining a position of the hand 17 with respect to the rear view imaging device; identifying a gesture of the hand 17; and controlling operation of the rear view imaging device (15) in dependence on a distance between at least a portion of the hand 17 and the rear view imaging device (15) being less than or equal to a predefined threshold distance, and the identified gesture. An orientation of the hand 17 may be determined using an input from a camera (7 see fig 1). The invention facilitates adjustment of the device (15).

Description

TECHNICAL FIELD

The present invention relates to a vehicle controller. Aspects of the invention relate to a controller for controlling operation of a rear view imaging device comprising a visual display area, to a vehicle comprising the controller, to a system for controlling operation of a rear view imaging device comprising the aforementioned controller and a time-of-flight image capture device, to a method of controlling operation of a rear view imaging device comprising a visual display, to a computer program product comprising instructions to carry out the aforementioned method, and to a computer readable data carrier having stored thereon instructions for carrying out the aforementioned method.

BACKGROUND

Current vehicles, in particular automobiles, are often provided with one or more rear view mirrors, and door-mirrors. Depending on the orientation of the mirrors relative to a vehicle occupant, a different image of an external environment towards the rear of the vehicle is displayed. The relative orientation of the mirrors may be varied by direct manipulation of the rear view imaging mirror or by operation of a physical switch configured to control the orientation of the rear view imaging mirror.

A disadvantage associated with the state of the art solution is that in certain circumstances it may be difficult or uncomfortable to adjust a rear view mirror or door mirror. Often a vehicle occupant, such as a driver of the vehicle, is required to adjust their position relative to the mirror to be able to adjust the relative position of the mirror. This may lead to difficulty in accurately configuring the mirror for use when the driver is seated in the driving position, since the image that is displayed on the mirror is to a certain extent dependent on the orientation of the mirror relative to the driver’s driving position. Often the driving position is sufficiently far away from the rear view imaging device that it is not possible to manipulate the position of the device from said the driving position. As such, the driver is often required to make several alterations on a ‘trial and error’ basis until the desired orientation with respect to the driving position is obtained. In order to mitigate this, control switches for controlling operation of the relevant rear view imaging devices are frequently provided in a body trim panel of the vehicle, located in closer proximity to a vehicle occupant. This solution means that the vehicle occupant is less likely to have to shift from their seated position. However, it is often difficult to operate these switches without the vehicle occupant diverting their gaze to the switch. This is often inconvenient, in particular whilst operating the vehicle.

It is an aim of at least certain embodiments of the present invention to address disadvantages associated with the prior art and in particular to facilitate control of rear view imaging devices.

SUMMARY OF THE INVENTION

Aspects and embodiments of the invention provide a method, a controller, a system, a vehicle, a computer program product and a computer readable data carrier as claimed in the appended claims.

According to an aspect of the present invention there is provided a method of controlling operation of a rear view imaging device for a vehicle, the rear view imaging device comprising a visual display area. The method may comprise detecting a hand of a vehicle occupant within a volume of space within a vehicle cabin; determining a position of the hand with respect to the rear view imaging device; identifying a gesture of the hand; and controlling operation of the rear view imaging device in dependence on a distance between at least a portion of the hand and the rear view imaging device being less than or equal to a predefined threshold distance, and the identified gesture.

In this way it is possible to selectively operate a rear view imaging device in dependence on a vehicle occupant’s hand being located within a predefined threshold distance of the desired imaging device. This enables the occupant of the vehicle to control operation of the rear view imaging device, without having to adjust their seated position. In addition, the rear view imaging device may be operated without the vehicle occupant having to divert their gaze. These are particularly advantageous benefits for a driver of a vehicle.

The method may comprise determining a position of the hand relative to a proximity boundary associated with the rear view imaging device. The proximity boundary may define a boundary offset from the rear view imaging device by the predefined threshold distance. The operation of the rear view imaging device may be controlled in dependence on at least a portion of the hand intersecting the proximity boundary.

In embodiments, the method may comprise determining if the hand is the vehicle occupant’s left or right hand and controlling operation of the rear view imaging device in dependence on whether the hand is the vehicle occupant’s left or right hand. This may comprise determining if the hand is orientated palm upwards or downwards within the volume of space relative to the rear view imaging device; and determining if the hand is the vehicle occupant’s left or right hand in dependence on the orientation of the hand. Analysis of the orientation of the hand provides a convenient way of determining whether the hand is a vehicle occupant’s left or right hand.

The method may comprise determining which vehicle occupant the hand belongs to and controlling operation of the rear view imaging device in dependence on which vehicle occupant the hand belongs to. In this way, advantageously, it is possible to selectively restrict control of the operation of the rear view imaging device in dependence on which vehicle occupant the hand belongs to. The method may further comprise controlling operation of the rear view imaging device in dependence on the hand belonging to a driver of the vehicle. This may be particularly advantageous where operation of the rear view imaging device by a vehicle occupant other than a driver of the vehicle may disrupt or otherwise inconvenience the driver.

The method may comprise determining a direction of entry of the hand into the volume of space relative to the rear view imaging device and determining which vehicle occupant the hand belongs to in dependence on the direction of entry of the hand into the volume of space. The direction of entry of the hand may be indicative of which vehicle occupant the hand belongs to, which in turn may be used for selectively restricting operation of the rear view imaging device to specific vehicle occupants.

The method may comprise obtaining image data of the hand within the volume of space; receiving a reflectance signal reflected from the hand; determining a distance of the hand from a designated origin in dependence on the reflectance signal; and determining the relative position of the detected hand with respect to the rear view imaging device in dependence on the distance of the hand relative to the designated origin, the obtained image data, and a known distance of the rear view imaging device relative to the designated origin. The time taken for the reflectance signal to be measured by a receiver is proportional to the distance of the hand from the sensor, and therefore provides a convenient way for distance information associated with the position of the hand to be obtained. In this way it is possible to determine the position of the hand relative to the rear view imaging device on the basis of a two-dimensional image of the hand relative to the rear view imaging device, and distance information of the hand. This significantly simplifies the hardware required to carry out the method, and in particular obviates the need for using a complex system of two or more cameras, each configured to obtain different perspective images of the hand relative to the rear view imaging device, from which the distance of the hand relative to the rear view imaging device may be determined.

In certain embodiments the designated origin may be coincident with the position of an image capture device.

In certain embodiments, the rear view imaging device may be associated with two or more different modes of operation, and the method may comprise identifying one of a plurality of different gestures; selecting the mode of operation in dependence on the identified gesture; and controlling operation of the rear view imaging device in accordance with the selected mode of operation. Optionally, the modes of operation may comprise a first mode of operation and a second mode of operation. In the first mode of operation, the visual display area may be configured to enable the vehicle occupant to see an external environment to the rear of the vehicle, and in the second mode of operation the visual display area may be configured to display information associated with one or more sensors of the vehicle. For example, the information associated with the one or more sensors of the vehicle may relate to GPS data, vehicle bearing data, atmospheric conditions external to the vehicle, and any other vehicle sensor data. This is advantageous since it increases the versatility of the rear view imaging device.

In certain embodiments, the rear view imaging device may comprise two or more different configuration settings, and the method may comprise: identifying one of a plurality of different gestures; and selecting the configuration setting in dependence on the identified gesture. The use of gestures to control configuration settings is beneficial since they enable the vehicle occupant to manipulate the configuration of the rear view imaging device without requiring that the vehicle occupant divert their gaze.

Optionally, the rear view imaging device may comprise a rear view mirror, and the method may comprise controlling an orientation of the rear view mirror with respect to the vehicle occupant in dependence on the identified gesture.

In certain embodiments the rear view imaging device may comprise a rear facing image capture device arranged to obtain an image of an external environment to the rear of the vehicle. The visual display area may be located within the vehicle cabin, and the method may comprise: controlling an orientation relative to the vehicle of the rear facing image capture device in dependence on the identified gesture. Different perspective images of the external environment to the rear of the vehicle may be displayed on the visual display located within the vehicle cabin by manipulating the orientation of the rear facing image capture device. In embodiments, the method may comprise adjusting a field of view of the rear facing image capture device in dependence on the identified gesture.

According to a further aspect of the invention there is provided a controller for controlling a rear view imaging device, the rear view imaging device comprising a visual display area. The controller may comprise an input configured to receive image data obtained by an image capture device, a processor and an output. The processor may be configured in use to: process the image data to recognise a hand of a vehicle occupant within the image data, the hand being located within a volume of space within a vehicle cabin within which the image data is obtained by the image capture device; to determine a position of the hand with respect to the rear view imaging device; and to identify a gesture of the hand from the image data. The output may be arranged in use to output a control signal for controlling operation of the rear view imaging device, in dependence on a distance between at least a portion of the hand and the rear view imaging device being less than or equal to a predefined threshold distance, and on the identified gesture. The controller and its embodiments benefit from the same advantages as set out in respect of the preceding aspects of the invention.

The processor may be configured in use to determine the relative position of the hand with respect to a proximity boundary associated with the rear view imaging device. The proximity boundary may define a boundary offset from the rear view imaging device by the predefined threshold distance. The output may be arranged in use to output the control signal in dependence on the position of at least a portion of the hand intersecting the proximity boundary.

The processor may be arranged in use to determine if the hand is the vehicle occupant’s left or right hand and the output may be arranged in use to output the control signal in dependence on whether the hand is the vehicle occupant’s left or right hand.

The processor may be arranged in use to determine if the hand is orientated palm upwards or downwards within the volume of space relative to the rear view imaging device and to determine if the hand is the vehicle occupant’s left or right hand in dependence on the orientation of the hand.

The processor may be arranged in use to determine the vehicle occupant the hand belongs to and the output may be arranged in use to output the control signal in dependence on which vehicle occupant the hand belongs to. Optionally, the output may be arranged in use to output the control signal in dependence on the hand belonging to a driver of the vehicle.

The processor may be arranged in use to determine a direction of entry of the hand into the volume of space relative to the imaging device and determining which vehicle occupant the hand belongs to in dependence on the direction of entry of the hand into the volume of space.

In certain embodiments the input may be configured to receive data from a time-offlight (ToF) image capture device comprising a sensor. The ToF image capture device may be arranged in use to obtain image data of the hand, to illuminate the hand within the volume of space, and to measure a time of return of a reflected illumination signal, the time of return of the reflected illumination signal being proportional to a distance of the hand from the sensor. The data may comprise the image data and the time of return of the reflected illumination signal. The processor may be arranged in use to determine the relative position of the hand with respect to the rear view imaging device in dependence on the determined distance of the hand from the sensor, the obtained image data, and a known distance of the rear view imaging device relative to the sensor, wherein the distance of the hand from the sensor is determined in dependence on the time of return of the reflected illumination signal. The ToF image capture device provides a convenient means for obtaining image data associated with image object distance data, and therefore simplifies determining the distance of the hand from the rear view imaging device.

In certain embodiments, the rear view imaging device may be associated with two or more different modes of operation, and the processor may be arranged in use to identify one of a plurality of different gestures. The output may be arranged in use to output a control signal selecting the mode of operation in dependence on the identified gesture and controlling operation of the rear view imaging device in accordance with the selected mode of operation. Optionally, the modes of operation may comprise a first mode of operation and a second mode of operation, in the first mode of operation, the visual display area may be configured to enable the vehicle occupant to see an external environment to the rear of the vehicle, and in the second mode of operation the visual display area may be configured to display information associated with one or more sensors of the vehicle.

In certain embodiments, the rear view imaging device may comprise two or more different configuration settings, the processor may be arranged in use to identify one of a plurality of different gestures and the output may be arranged in use to output a control signal for selecting the configuration setting in dependence on the identified gesture.

Optionally, the rear view imaging device may comprise a rear view mirror, and the output may be arranged in use to output a control signal for controlling an orientation of the rear view mirror with respect to the vehicle occupant in dependence on the identified gesture.

In certain embodiments the rear view imaging device may comprise a rear facing image capture device arranged to obtain an image of an external environment to the rear of the vehicle. The visual display area may be located within the vehicle cabin, and the output may be arranged to: output a control signal for controlling an orientation of the rear facing image capture device relative to the vehicle in dependence on the identified gesture. Additionally or alternatively, the output may be arranged to: output a control signal for adjusting a field of view of the rear facing image capture device in dependence on the identified gesture.

In accordance with yet a further aspect of the invention, there is provided a system for controlling operation of a rear view imaging device comprising a visual display. The system may comprise the aforementioned controller and an image capture device. Optionally, the image capture device may comprise a time of flight (ToF) image capture device.

In accordance with yet a further aspect of the invention, there is provided a computer program product comprising instructions for carrying out the aforementioned method.

The computer program product may comprise instructions, which when executed on a processor, configure the processor to: detect a hand of a vehicle occupant within a volume of space within the vehicle cabin; determine the relative position of the detected hand with respect to the rear view imaging device; and control operation of the rear view imaging device in dependence on a distance between at least a portion of the hand and the imaging device being less than or equal to a predefined threshold distance.

In accordance with yet a further aspect of the invention, there is provided a computer readable data carrier having stored thereon instructions for carrying out the aforementioned method. Optionally, the computer readable data carrier comprises a non-transitory computer readable data carrier.

The data carrier may comprise instructions, which when executed on a processor, configure the processor to: detect a hand of a vehicle occupant within a volume of space within the vehicle cabin; determine the relative position of the detected hand with respect to the rear view imaging device; and control operation of the rear view imaging device in dependence on a distance between at least a portion of the hand and the imaging device being less than or equal to a predefined threshold distance.

In accordance with yet a further aspect of the invention, there is provided a vehicle configured to carry out the aforementioned method.

In accordance with yet a further aspect of the invention, there is provided a vehicle comprising the aforementioned controller, or the aforementioned system.

Within the scope of this application it is expressly intended that the various aspects, embodiments, examples and alternatives set out in the preceding paragraphs, in the claims and/or in the following description and drawings, and in particular the individual features thereof, may be taken independently or in any combination. That is, all embodiments and/or features of any embodiment can be combined in any way and/or combination, unless such features are incompatible. The applicant reserves the right to change any originally filed claim or file any new claim accordingly, including the right to amend any originally filed claim to depend from and/or incorporate any feature of any other claim although not originally claimed in that manner.

BRIEF DESCRIPTION OF THE DRAWINGS

One or more embodiments of the invention will now be described, by way of example only, with reference to the accompanying drawings, in which:

Figure 1 is a schematic illustration of a front portion of a vehicle cabin having a camera having a field of view arranged to obtain image data of a vehicle occupant’s hand within a volume of space within a vehicle cabin;

Figure 2 is a schematic illustration of a controller configured to output a control signal for controlling operation of a rear view imaging device located in the vehicle cabin of Figure 1;

Figure 3 is a process flow chart outlining a method for controlling operation of a rear view imaging device from within a vehicle cabin, in dependence on the proximity of a vehicle occupant’s hand to the rear view imaging device, using the camera of Figure 1;

Figure 4 is a schematic illustration highlighting the principle of operation of a Time-ofFlight (ToF) camera, which may be used to determine the position of a vehicle occupant’s hand within the vehicle cabin of Figure 1;

Figures 5a and 5b are schematic illustrations showing a three-dimensional point cloud of a vehicle occupant’s hand generated using the ToF camera of Figure 4;

Figures 6a and 6b are schematic illustrations showing two examples of a rear view imaging device, whose operation is controlled in dependence on the proximity of the vehicle occupant’s hand to the imaging device;

Figures 7a and 7b are schematic illustrations showing different modes of operation of the rear view imaging device, in dependence on the proximity of the vehicle occupant’s hand to the rear view imaging device, and in dependence on the identified gesture; and

Figure 8 is a schematic illustration of a vehicle comprising the camera of Figure 1, and the controller of Figure 2.

DETAILED DESCRIPTION

Figure 1 is a perspective view of a portion of the vehicle cabin 1, and in particular shows the driver 3 sitting in the driver’s seat 5. An image capture device in the form of a camera 7, having a field of view 9 delineated in Figure 1 by lines 11, is shown located in the cabin roof. Optionally, the camera 7 may comprise a Time-of-flight (ToF) camera. The camera 7 is arranged to image objects located within the camera’s field of view 9. The field of view defines a volume of space within the vehicle cabin 1 within which objects are imaged by the camera 7. The camera 7 is arranged such that at least a portion of a rear view imaging device 15 lies within the camera’s field of view 9. The rear view imaging device 15 may comprise any suitable apparatus which enables an occupant of the vehicle 43 to see an image of an external environment to the rear of the vehicle 43. The rear view imaging device 15 may additionally comprise any suitable apparatus which enables the perspective of the displayed image to be altered. In certain embodiments, a rear view imaging device 15 may be entirely housed within the vehicle cabin 1. In alternative embodiments, the rear view imaging device 15 may comprise one or more portions which are housed on an exterior surface of the vehicle 43. The vehicle cabin 1 may be comprised in the vehicle 43 of Figure 9.

The rear view imaging device 15 may comprise a visual display area 16 upon which the image is displayed. In certain embodiments, this visual display area 16 may comprise a reflective surface, such as a mirrored surface, which is configured to reflect an incident image of an external environment toward a rear of the vehicle 43 to an occupant of the vehicle 43. In certain embodiments the visual display area 16 may comprise, but is not limited to: a rear view mirror configured to enable the vehicle occupant to see rearward through a rear windshield of the vehicle 43; and/or a door mirror attached to an exterior surface of the vehicle 43 configured to enable the vehicle occupant to see rearward at an exterior side of the vehicle 43. In other embodiments, the visual display area 16 may comprise a display screen, which is configured to display image data obtained by an image capture device arranged to obtain images of an external environment towards a rear of the vehicle 43. The image data may be collected by any suitable image capture device (not shown), such as a rear facing camera. In certain embodiments, the display screen may be configured to display additional information associated with the vehicle 43, which may be used by a vehicle occupant to enhance a driving experience. Such information may include, but is not limited to, global positioning system (GPS) navigational information, vehicle velocity, fuel data, and any other information associated with the vehicle 43 including vehicle sensor data. In certain embodiments, a portion of the visual display area 16 may comprise a reflective surface and a second portion may comprise a display screen displaying the aforementioned information.

In certain embodiments, as shown in Figure 1, at least a portion of the rear view imaging device 15 may be mounted on an interior of the front windshield of the vehicle 43 within the vehicle cabin 1. Similarly, at least a portion of the rear view imaging device 15 may be mounted on or within a body panel 13 of the vehicle 43, where the body panel may additionally lie within the field of view 9 of the camera 7.

In certain embodiments, the camera 7 may be operatively coupled to a controller 19 (shown in Figure 2), and the controller 19 may be configured to receive image data obtained by the camera 7 and to output a control signal for controlling operation of the rear view imaging device 15 to the rear view imaging device 15, in dependence on an analysis of the received image data. This enables selective operation of the rear view imaging device 15. Within the present context, operation of the rear view imaging device 15 may relate to controlling the image displayed on the visual display area 16 of the rear view imaging device 15. In certain embodiments where the display area 16 comprises a reflective surface, this may comprise moving at least a portion of the visual display area 16 relative to a vehicle occupant, to alter the image displayed to the vehicle occupant. This may be achieved by the provision of a mechanical motor as part of the rear view imaging device 15 configured to move the display area 16 upon receipt of a suitable control signal. In other embodiments, it may comprise moving an image capture device configured to obtain image data of an exterior of the vehicle 43, and which image data is displayed on the visual display area 16. This again may be achieved by the provision of a mechanical motor configured to physically move the image capture device.

In yet further embodiments, the operation may comprise altering the information displayed on the visual display area 16. This may comprise displaying available GPS navigational information, which may be displayed in addition to an image of the external environment to the rear of the vehicle 43. In such embodiments, the rear view imaging device 15 may be configured to receive information from a suitable electronic hardware memory located within the vehicle 43, such as from a navigation system and, upon receiving a suitable control signal, is able to display the information on the visual display area 16.

In the ensuing description the term Operation mechanism’ may be used to refer to any mechanism for controlling operation of the rear view imaging device 15 including the previously described embodiments.

Figure 2 provides a functional overview of the controller 19. The controller 19 may be functionally embedded into an existing electronic control unit of the vehicle 43. The controller 19 may be provided with an input 21 and an output 23. The input 21 may be configured to receive image data obtained by the camera 7, and the output 23 may be configured to output a control signal to the rear view imaging device 15, and specifically to an operation mechanism associated with the rear view imaging device 15. For the purposes of the present description the operation mechanism will be taken to be an inherent component of the rear view imaging device 15, such that any reference to outputting a control signal to the rear view imaging device 15 may be understood as a control signal output to the operation mechanism of the rear view imaging device 15. The controller 19 may additionally comprise a processor 25 arranged to analyse image data received from the camera 7, to identify image objects such as the hand 17 of a vehicle occupant and a gesture of the hand within the obtained image data, and to generate and output control signals for operating the given rear view imaging device 15, in dependence on the relative position of the hand 17 with respect to the rear view imaging device 15 and the identified gesture.

In use, as a vehicle occupant’s hand is obtained by the camera 7, and its position relative to a given rear view imaging device 15 is determined and the gesture identified by the controller 19, typically by the processor 25 of the controller 19, the rear view imaging device 15 may be operated by the controller 19 via a control signal output to the rear view imaging device 15. In this way, the rear view imaging device 15 may be operated without requiring physical contact between the vehicle occupant’s hand 17 and the rear view imaging device 15.

In certain embodiments, the controller 19 may be configured in use to output the control signal in dependence on a distance between at least a portion of a vehicle occupant’s hand 17 and the rear view imaging device 15 being less than or equal to a predefined threshold distance. For example, as the camera 7 obtains image data of a vehicle occupant’s hand, such as the driver’s hand 17, the controller 19 may be configured to identify the image of the hand and the gesture within the received obtained image data. The relative position of the imaged hand with respect to the rear view imaging device 15 and the gesture may then be determined, from the obtained image data. In order to identify an image of a hand, the controller 19, and specifically the processor 25 may be configured with image recognition software configured to identify a vehicle occupant’s hand 17 and the associated gesture located within the camera’s field of view 9, from obtained image data.

Figure 3 is a process flow chart outlining the method used in accordance with certain embodiments of the invention, to control operation of the rear view imaging device 15 in dependence on the proximity of a vehicle occupant’s hand to the rear view imaging device 15 and the identified gesture, using the camera 7 in operative communication with the controller 19. The method is initiated by the camera 7 obtaining image data within the vehicle cabin 1, at step 301. In certain embodiments the camera 7 may be configured to continuously obtain the image data, or to periodically obtain image data at a predefined frequency. The obtained image data may be forwarded to the controller 19 for analysis where, at step 303, obtained image data is analysed to identify a vehicle occupant’s hand 17 within the obtained image data. As mentioned previously, this may comprise the use of image recognition software. Once a vehicle occupant’s hand 17 has been identified within the obtained image data, the position of the hand 17 is determined relative to the rear view imaging device 15, at step 305. The position of the hand 17 relative to the rear view imaging device 15 and the gesture may be determined by the processor 25. At step 307 it is determined if at least a portion of the hand 17 lies at a distance that is less than or equal to a predefined threshold distance from the rear view imaging device 15. If it is determined that no portion of the hand lies within the predefined threshold distance, then the processor 25 continues to analyse received obtained image data, and the method returns to step 303. If instead it is determined by the processor 25 that at least a portion of the identified hand lies within the predefined threshold distance of the rear view imaging device 15, then the processor proceeds by analysing the obtained image data in order to identify a gesture which the vehicle occupant’s hand 17 is performing, at step 308. In accordance with a previously described embodiment, this may comprise the use of image recognition software. Upon identification of a gesture, the processor generates a control signal for output to the rear view imaging device 15, at step 309. Upon receipt of the control signal, at step 310, the rear view imaging device 15 is operated.

In certain embodiments the predefined threshold distance may relate to a few centimetres, for example any distance within the range of 1cm to 10cm, including 1cm and 10cm. In certain embodiments the predefined threshold may delineate a control proximity boundary surrounding and offset from the rear view imaging device 15 by the predetermined threshold distance, which when intersected by at least a portion of the vehicle occupant’s hand causes the controller 19 to generate the control signal for output to the rear view imaging device 15.

The control proximity boundary may be geometrically shaped. For example, the control proximity boundary may be box-shaped, or spherically shaped. Effectively, the control proximity boundary relates to a volume of space offset from the rear view imaging device 15 by the predefined threshold distance. In dependence on any portion of the control proximity boundary being intersected by at least a portion of the vehicle occupant’s hand, the controller generates the control signal for operating the rear view imaging device 15. It is to be appreciated that not all of the portions of the control proximity boundary need to be offset from the rear view imaging device 15 by the predefined threshold distance. For example, where the control proximity boundary is box-shaped (e.g. cube shaped), it is to be appreciated that some faces of the cube may not be offset from the rear view imaging device 15 by the predefined threshold distance.

In certain embodiments, in order to enable the position of the hand 17 to be determined relative to the rear view imaging device 15, the camera 7 may relate to a Time-of-Flight (ToF) camera, in which each obtained image pixel is associated with a distance on the basis of a time of return of a reflected illumination signal. To achieve this, the ToF camera may be configured with an illumination source arranged to illuminate the camera’s field of view. The incident illumination signal is subsequently reflected by objects present in the camera’s field of view, and the time of return of the reflected illumination signal is measured. In this way it is possible to associate a distance measurement to each imaged object. The illumination signal may relate to any electro-magnetic signal, and need not be comprised in the visible spectrum. For example, in certain embodiments the illumination signal may operate in the infrared spectrum.

In those embodiments comprising a ToF camera 7, the controller 19, and specifically the input 21 may be configured to receive both camera image data and image object distance information data from the ToF camera 277. This enables the controller 19, and more specifically the processor 25 to determine the position of the vehicle occupant’s hand 17 relative to a rear view imaging device 15 from the received data.

Figure 4 is a schematic diagram illustrating the principle of operation of a ToF camera

27. A modulated illumination source 29 is used to illuminate a desired target 31. The incident illumination 33 is reflected by the target 31 and captured on a sensor 35 comprising an array of pixels. However, whilst simultaneously capturing the reflected modulated light 37, the pixels of the sensor 35 also capture visible light reflected from the target. Since the illumination signal is modulated 33, it may be distinguished from the visible light reflected from the target 31, which enables the time of flight of the modulated illumination signal to be measured. The time of flight taken for the modulated illumination signal to be incident on the target 31 and reflected back to the sensor 35 is measured when it is incident on the sensor 35. In this way, each captured image pixel may be associated with a distance of the corresponding image object on the basis of the measured time of flight required for the reflected modulated illumination signal 37 to be measured by the sensor 35. More specific details regarding operation of ToF cameras are widely available in the art, and for this reason a more detailed discussion is not necessary for present purposes.

Where the camera 7 of Figure 1 comprises a ToF camera 27, it is possible to generate a three-dimensional point cloud of the vehicle occupant’s hand located within the camera’s field of view 9. Figures 5a and 5b illustrate an example of a threedimensional point cloud 39 of the vehicle occupant’s hand 17, generated using the ToF camera 27. In certain embodiments the controller 19 may be configured to generate the three-dimensional point cloud using the image data and image object distance information received from the ToF camera 27. Figure 5a shows a point cloud 39 of the vehicle occupant’s hand 17 as it is approaching a rectangular-shaped control proximity boundary 41. In Figure 5b a portion of the point cloud 39 of the vehicle occupant’s hand 17 is intersecting a portion of the control proximity boundary 41. In this event, and as mentioned previously, the controller 19 is configured to generate a control signal for operating the rear view imaging device 15.

In order to enable the position of the vehicle occupant’s hand 17 to be determined relative to a rear view imaging device 15, the position of the rear view imaging device 15 relative to the ToF camera may be determined. Again, this may be done using image recognition software. Since the position of the rear view imaging device 15 relative to the ToF camera 27 is known, and the position of the vehicle occupant’s hand 17 relative to the ToF camera 27 is known, the position of the vehicle occupant’s hand 17 relative to the rear view imaging device 15 may be determined using trigonometry. In certain embodiments, and in order to facilitate computation during use, the controller 19 may be provided with distance information of rear view imaging device 15 relative to the ToF camera 27 during an initial configuration of the ToF camera 27. This distance information may be stored and accessed for subsequent use when it’s needed. This facilitates subsequent computation of the position of the hand relative to the rear view imaging device 15, since only the distance of the vehicle occupant’s hand 17 with respect to the ToF camera 27, and the position relative to the known position of the rear view imaging device 15 requires calculation, both of which may be obtained from data obtained by the ToF camera 27.

Figures 6a and 6b show an example use scenario wherein, in response to at least a portion of a vehicle occupant’s hand 17 intersecting a control proximity boundary 61, the rear view imaging device 15 associated with the control proximity boundary 61 is configured to be operated. In the illustrated examples the rear view imaging device 15 may comprise a rear view mirror 62 configured to enable the vehicle occupant to see rearward through a rear windshield of the vehicle 43. Figure 6a shows the scenario where the vehicle occupant’s hand 17 is at a position where at least a portion of the hand intersects the control proximity boundary 61. As a result, the rear view mirror 62 is configured to rotate or otherwise move. This causes the image which is incident upon the reflective surface 64 of the associated rear view mirror 62 to change which, as a result, causes the image which is presented to the vehicle occupant to also change. Figure 6b shows a further scenario where the operation which the rear view mirror 62 undergoes is dependent on a gesture or movement which is performed by the vehicle occupant’s hand 17. For example, if the vehicle occupant’s hand 17 moves left in relation to the rear view mirror 62, the mirror 62 may be configured to rotate clockwise, whereas if the vehicle occupant’s hand 17 moves right in relation to the rear view mirror 62, the mirror 62 may be configured to rotate anticlockwise.

Figures 7a and 7b show further example use scenarios, in which the rear view imaging device 15 comprises a display screen 71 which is configured to display image data 73 relating to the external environment to the rear of the vehicle 43, provided by an image capture device (not shown). The display screen is also configured to display global positional system navigational information 75 relating to the vehicle 43. In Figure 7a, the display screen 71 is configured to display only image data 73 relating to the external environment to the rear of the vehicle 43. In Figure 7a the vehicle occupant’s hand 17 is positioned such that no part of the hand intersects with the control proximity boundary 77. As the vehicle occupant’s hand 17 intersects the control proximity boundary 77, as shown in Figure 7b, the image displayed on the display screen 71 may be configured to change to simultaneously display image data 73 relating to the external environment to the rear of the vehicle 43 and also to display global positional system navigational information 75 relating to the vehicle 43.

In certain embodiments it is envisaged that once the rear view imaging device 15 has been operated to alter the image displayed on the visual display area 16, the image displayed image remains in this altered configuration for a predefined period of time irrespective of the position of the vehicle occupant’s hand relative to the rear view imaging device 15. For example, when a rear view imaging device 15 is operated such that image data 73 relating to the external environment to the rear of the vehicle 43 and global positional system navigational information 75 relating to the vehicle 43 are configured to be simultaneously displayed, should the vehicle occupant then retract their hand such that it no longer lies within the predefined threshold distance, then the image displayed on the visual display area 16 will remain in this configuration for the predefined period of time before transitioning to its initial display. The predefined period of time may relate to any arbitrary period of time and may be dependent on the specific rear view imaging device 15 and the image which is being displayed. For example, GPS navigational information may remain displayed for several minutes after the hand has been retracted, whilst information relation to the velocity of the vehicle 43 may only remain displayed for several seconds after retraction of the vehicle occupant’s hand.

In a further embodiment, it is envisaged that once the rear view imaging device 15 has been operated to alter image displayed on the visual display area 16, this altered image continues to be displayed until a further operation of the rear view imaging device 15 is enacted. For example, if the image displayed on the visual display area 16 is adjusted to display a different view of the external environment to the rear of the vehicle 43, this adjusted view may remain displayed until such a time that the rear view imaging device 15 is operated again. In this way it is possible to adjust the image displayed on the visual display area 16 to adhere to a vehicle occupant’s preferences without reverting to a less preferred image once the occupant’s hand 17 is removed.

In a further embodiment, the controller 19 may be configured to determine from an analysis of the received camera image data whether the vehicle occupant’s hand 17 is the occupant’s left or right hand. The controller 19 may then be configured to output the control signal in dependence on whether the hand 17 is the occupant’s left or right hand. In this way it is possible to restrict operation of a desired rear view imaging device 15 within the vehicle cabin 1 dependent on whether a vehicle occupant’s left or right hand is attempting to operate the associated rear view imaging device 15.

In certain embodiments, the controller 19 may be configured to determine if the vehicle occupant’s hand 17 is orientated palm upwards or downwards relative to the camera 7, and determining if the hand 17 is the vehicle occupant’s left or right hand in dependence on whether the hand is orientated palm upwards or downwards. This may be determined on the basis of the reflectance signal from the hand 17, and by image object analysis. The skin texture of a palm of a hand is different to the skin texture of the back of a hand, and as a result the amount of incident light absorbed by the palm differs to the amount of incident light absorbed by the back of the hand. Accordingly, by configuring the controller to analyse the intensity of the reflected signal, which is indicative of the amount of incident illumination absorbed by the hand, it is possible for the controller to determine whether the hand is orientated palm upwards or downwards.

In certain embodiments, the controller 19 may be configured to determine which vehicle occupant the imaged hand belongs to, for example, whether the imaged hand 17 belongs to a driver 3 of the vehicle or to a passenger. Operation of the rear view imaging device 15 may then be controlled in dependence on which vehicle occupant the hand belongs to. For example, operation of the rear view imaging device 15 may be controlled in dependence on the hand 17 belonging to the driver 3 of the vehicle. This helps to prevent accidental operation of a rear view imaging device 15 by an unauthorised vehicle occupant. One non-limiting way in which the controller 19 may determine which vehicle occupant the hand belongs to, is by monitoring and determining a direction of entry of the hand into the camera’s field of view 9 relative to the rear view imaging device 15. This may be achieved from an analysis by the controller 19 of image data obtained by the camera 7. The direction of entry of the hand 17 into the camera’s field of view 9 may be indicative of where the vehicle occupant is seated in relation to the rear view imaging device 15, and therefore provides a good assumption regarding which vehicle occupant the hand 17 belongs to.

In a further embodiment, the controller 19 may be configured to determine from an analysis of the received camera image data whether the vehicle occupant’s hand 17 is performing a predefined gesture. The controller 19 may then be configured to output the control signal in dependence on whether the hand 17 is performing the predefined gesture. In this way, the controller 19 may able to restrict the operation of the rear view imaging device 15 not only in dependence on the relative position of the hand with respect to the rear view imaging device, but also in dependence on the predefined gesture. This further helps to prevent accidental operation of a rear view imaging device, and for certain rear view imaging devices may be desirable, in particular where operation of such rear view imaging devices may have safety consequences. Additionally, as discussed above with reference to Figure 6b, the use of different gestures may relate to different operations associated with the rear view imaging device 15. For example, if the vehicle occupant moves their hand in one direction within the camera’s 7 field of view, the rear view imaging device 15 may be configured to rotate clockwise about an axis of symmetry, whereas if the opposite movement is performed, the rear view imaging device 15 may be configured to rotate anti-clockwise. Similarly, performing a particular gesture may cause global positioning system information to be displayed or removed from the visual display area 16, whereas performing a different gesture may cause information regarding the vehicle’s 43 velocity to be displayed, in accordance with an embodiment described above.

In a further embodiment, the controller 19 may be configured to determine from an analysis of the received camera image data whether the vehicle occupant’s hand 17 is performing one of a plurality of predefined gestures. The controller 19 may then be configured to output the control signal in dependence on which predefined gesture the hand 17 is performing. The use of different gestures may be used to control different operations associated with the rear view imaging device 15. For example, the use of different gestures may be used to vary the orientation of the rear view imaging device 15, e.g. if the vehicle occupant moves their hand in one direction within the camera’s 7 field of view, the rear view imaging device 15 may be configured to rotate clockwise about an axis of symmetry, whereas if the opposite movement is performed, the rear view imaging device 15 may be configured to rotate anti-clockwise. Similarly, performing a particular gesture may cause GPS information to be displayed on the visual display area 16. In this way, the versatility of the rear view imaging device 15 may be improved.

Whilst the preceding embodiments of the invention have been described within the context of a ToF camera, it is to be appreciated that alternative camera configurations may be used in accordance with the herein described embodiments. Any configuration of cameras may be used that enables image data of a hand relative to a rear view imaging device 15 to be obtained, and the position of the hand relative to the rear view imaging device 15 to be determined. For example, a configuration of two or more cameras each configured to enable a different perspective image of the hand relative to the rear view imaging device 15 to be obtained may also be used. In such an arrangement the different perspective images of the hand relative to the rear view imaging device 15 would enable the controller to determine the position of the hand with respect to the rear view imaging device 15 by triangulation.

Similarly, in an alternative embodiment, the ToF camera of the preceding embodiments may be replaced by a conventional camera, in combination with an optical ruler, such as a LIDAR for example. In such an embodiment the LIDAR provides the image object distance information, whilst the camera provides image data. The controller may be configured in such embodiments to analyse the LIDAR data in combination with the obtained image data in order to determine the position of the vehicle occupant’s hand relative to the rear view imaging device 15.

It is to be appreciated that many modifications may be made to the above examples and embodiments without departing from the scope of the present invention as defined in the accompanying claims.

Claims

1. A method of controlling operation of a rear view imaging device of a vehicle, the rear view imaging device comprising a visual display area, the method comprising:

detecting a hand of a vehicle occupant within a volume of space within a vehicle cabin;

determining a position of the hand with respect to the rear view imaging device;

identifying a gesture of the hand; and controlling operation of the rear view imaging device in dependence on a distance between at least a portion of the hand and the rear view imaging device being less than or equal to a predefined threshold distance, and the identified gesture.

2. The method of claim 1, comprising:

determining the position of the hand with respect to a proximity boundary associated with the rear view imaging device, the proximity boundary defining a boundary offset from the rear view imaging device by the predefined threshold distance; and controlling operation of the rear view imaging device in dependence on the position of at least a portion of the hand intersecting the proximity boundary.

3. The method of claim 1 or 2, comprising:

determining if the hand is the vehicle occupant’s left or right hand; and controlling operation of the rear view imaging device in dependence on whether the hand is the vehicle occupant’s left or right hand.

4. The method of any preceding claim, comprising:

determining if the hand is orientated palm upwards or downwards relative to the rear view imaging device;

determining if the hand is the vehicle occupant’s left or right hand in dependence on orientation of the hand; and controlling operation of the rear view imaging device in dependence on whether the hand is the vehicle occupant’s left or right hand.

5. The method of any preceding claim, comprising:

determining which vehicle occupant the hand belongs to; and controlling operation of the rear view imaging device in dependence on which vehicle occupant the hand belongs to.

6. The method of claim 5, comprising:

controlling operation of the rear view imaging device in dependence on the hand belonging to a driver of the vehicle.

7. The method of claim 5 or 6, comprising:

determining the direction of entry of the hand into the volume of space relative to the rear view imaging device; and determining which vehicle occupant the hand belongs to in dependence on the direction of entry of the hand into the volume of space.

8. The method of any preceding claim, comprising:

obtaining image data of the hand within the volume of space;

receiving a reflectance signal reflected from the hand;

determining a distance of the hand from a designated origin in dependence on the reflectance signal; and determining the relative position of the hand with respect to the rear view imaging device in dependence on the distance of the hand relative to the designated origin, the obtained image data, and a known distance of the rear view imaging device relative to the designated origin.

9. The method of claim 8, wherein the designated origin is coincident with a position of an image capture device.

10. The method of any preceding claim, wherein the rear view imaging device is associated with two or more different modes of operation, and the method comprises:

identifying one of a plurality of different gestures;

selecting the mode of operation in dependence on the identified gesture; and controlling operation of the rear view imaging device in accordance with the selected mode of operation.

11. The method of claim 10, wherein the rear view imaging device comprises a first mode of operation and a second mode of operation, wherein in the first mode of operation the visual display area is configured to enable the vehicle occupant to see an external environment to the rear of the vehicle, and in the second mode of operation the visual display area is configured to display information associated with one or more sensors of the vehicle; and the method comprises selecting the first mode of operation or the second mode of operation in dependence on the identified gesture.

12. The method of any preceding claim, wherein the rear view imaging device comprises two or more different configuration settings, and the method comprises:

identifying one of a plurality of different gestures; and selecting the configuration setting in dependence on the identified gesture.

13. The method of any preceding claim, wherein the rear view imaging device comprises a rear view mirror, and the method comprises:

controlling an orientation of the rear view mirror with respect to the vehicle occupant in dependence on the identified gesture.

14. The method of any one of claims 1 to 12, wherein the rear view imaging device comprises a rear facing image capture device arranged to capture an image of an external environment to the rear of the vehicle, and the visual display area is located within the vehicle cabin, the method comprising:

controlling an orientation relative to the vehicle of the rear facing image capture device in dependence on the identified gesture; and/or adjusting a field of view of the rear facing image capture device in dependence on the identified gesture.

15. A controller for controlling a rear view imaging device, the rear view imaging device comprising a visual display area, the controller comprising:

an input configured to receive image data obtained by an image capture device;

a processor configured in use to:

process the image data to recognise a hand of a vehicle occupant within the image data, the hand being located within a volume of space within a vehicle cabin within which the image data is obtained by the image capture device;

to determine a position of the hand with respect to the rear view imaging device;

to identify a gesture of the hand from the image data; and an output arranged in use to output a control signal for controlling operation of the rear view imaging device, in dependence on a distance between at least a portion of the hand and the rear view imaging device being less than or equal to a predefined threshold distance, and the identified gesture.

16. The controller of claim 15, wherein the processor is arranged in use to determine the position of the hand with respect to a proximity boundary associated with the rear view imaging device, the proximity boundary defining a boundary offset from the rear view imaging device by the predefined threshold distance; and the output is arranged in use to output the control signal in dependence on the position of at least a portion of the hand intersecting the proximity boundary.

17. The controller of claim 15 or 16, wherein the processor is arranged in use to determine if the hand is the vehicle occupant’s left or right hand; and the output is arranged in use to output the control signal in dependence on whether the hand is the vehicle occupant’s left or right hand.

18. The controller of claim 17, wherein the processor is arranged in use to determine if the hand is orientated palm upwards or downwards within the volume of space relative to the rear view imaging device; and to determine if the hand is the vehicle occupant’s left or right hand in dependence on the orientation of the hand.

19. The controller of any one of claims 15 to 18, wherein the processor is arranged in use to determine which vehicle occupant the hand belongs to; and the output is arranged in use to output the control signal in dependence on which vehicle occupant the hand belongs to.

20. The controller of claim 19, wherein the output is arranged in use to output the control signal in dependence on the hand belonging to a driver of the vehicle.

21. The controller of claim 19 or 20, wherein the processor is arranged in use to determine a direction of entry of the hand into the volume of space relative to the rear view imaging device; and to determine which vehicle occupant the hand belongs to in dependence on the direction of entry.

22. The controller of any one of claims 15 to 21, wherein:

the input is configured to receive data from a time-of-flight (ToF) image capture device comprising a sensor, the ToF image capture device being arranged in use to obtain image data of the hand, to illuminate the hand, and to measure a time of return of a reflected illumination signal, the time of return of the reflected illumination signal being proportional to a distance of the hand from the sensor, the data comprising the obtained image data and the time of return of the reflected illumination signal; and wherein the processor is arranged in use to determine the position of the hand with respect to the rear view imaging device in dependence on the determined distance of the hand from the sensor, the obtained image data, and a known distance of the rear view imaging device relative to the sensor, wherein the distance of the hand from the sensor is determined in dependence on the time of return of the reflected illumination signal.

23. The controller of any one of claims 15 to 22, wherein the rear view imaging device is associated with two or more different modes of operation, and the processor is arranged in use to identify one of a plurality of different gestures; and the output is arranged in use to output a control signal for selecting and controlling the mode of operation in dependence on the identified gesture.

24. The controller of claim 23, wherein the rear view imaging device comprises a first mode of operation and a second mode of operation, wherein in the first mode of operation the visual display area is configured to enable the vehicle occupant to see an external environment to the rear of the vehicle, and in the second mode of operation the visual display area is configured to display information associated with one or more sensors of the vehicle; and the output is arranged in use to output the control signal for selecting the first mode of operation or the second mode of operation in dependence on the identified gesture.

25. The controller of any one of claims 15 to 24, wherein the rear view imaging device comprises two or more different configuration settings, the processor is arranged in use to identify one of a plurality of different gestures; and the output is arranged in use to output the control signal for selecting the configuration setting in dependence on the identified gesture.

26. The controller of any one of claims 15 to 25, wherein the rear view imaging device comprises a rear view mirror, and the output is arranged in use to output a control signal for controlling an orientation of the rear view mirror with respect to the vehicle occupant in dependence on the identified gesture.

27. The controller of any one of claims 15 to 25, wherein the rear view imaging device comprises a rear facing image capture device arranged to capture an image of an external environment to the rear of the vehicle, the visual display area is located within the vehicle cabin, and the output is arranged to:

output a control signal for controlling an orientation of the rear facing image capture device relative to the vehicle in dependence on the identified gesture; and/or output a control signal for adjusting a field of view of the rear facing image capture device in dependence on the identified gesture.

28. A system for controlling operation of a rear view imaging device for a vehicle, the

5 system comprising: the controller of any one of claims 15 to 27 in combination with a time-of-flight (ToF) image capture device.

29. A computer program product comprising instructions for carrying out the method of any one of claims 1 to 14.

30. A computer-readable data carrier having stored thereon instructions for carrying out the method of any one of claims 1 to 14.

31. A vehicle configured to carry out the method of any one of claims 1 to 14.

32. A vehicle comprising the controller of any one of claims 15 to 27.