DE102018218876A1 - Vehicle control device - Google Patents

Abstract

A method for enabling the use of a user-operated control device in a vehicle cabin. The method includes detecting a hand of a vehicle occupant within a volume of space in the vehicle cabin; determining a position of the hand with respect to the user-operated control device; determining whether the hand belongs to a first vehicle occupant or a second vehicle occupant; and enabling use of the user-operated control device in response to a distance between at least a portion of the hand and the user-operated control device that corresponds at most to a predefined threshold distance and depending on which vehicle occupant the hand belongs to.

Description

TECHNICAL AREA

The present disclosure relates to a vehicle control device. Aspects of the invention relate to the control device for enabling use of a user-operated control device within a vehicle cabin that enables the use of a user-operated control device within a vehicle cabin, a vehicle that is in use to perform the aforementioned method, and / or the previously said control device comprising, a computer program product comprising instructions for carrying out the aforementioned method, a computer-readable data carrier with instructions stored thereon for carrying out the aforementioned method, and a system comprising the aforementioned control device and a time-of-flight camera.

STATE OF THE ART

Modern vehicles, particularly motor vehicles, are often provided with one or more user-operated control devices, such as switches, infotainment systems, displays, and other controls placed within the vehicle cabin, configured to allow a user to change various settings of the vehicle. Each occupant of the vehicle may operate one of these user-operated control devices, provided that they are within range of the control devices. However, it is not always desirable for all vehicle occupants to have this freedom. For example, when a child is sitting in a passenger seat, it may not be desirable to be able to operate control devices that change important vehicle settings. This can be very annoying for a driver.

Accordingly, it is an object of at least certain embodiments of the present invention to overcome or ameliorate at least some of the disadvantages associated with the prior art.

BRIEF SUMMARY OF THE INVENTION

Aspects and embodiments of the invention provide a method, a controller, a system, a vehicle, and a computer-readable medium as claimed in the appended claims.

In accordance with one aspect of the present invention, a method for enabling the use of a user-operated control device in a vehicle cabin is provided. The method may include: detecting a hand of a vehicle occupant within a volume of space in the vehicle cabin; Determining a position of the hand with respect to the user-operated control device; Determining whether the hand belongs to a first vehicle occupant or to a second vehicle occupant; and allowing use of the user-operated control device in response to a distance between at least a portion of the hand and the user-operated control device corresponding to at most a predefined threshold distance, and depending on which vehicle occupant the hand belongs. In this way, it is advantageously possible to selectively restrict the use of a user-operated control device depending on which vehicle occupant attempts to use the user-operated control device. The use of the control device may be limited to only one authorized vehicle occupant. For example, the use of the control device may be limited solely to a driver of the vehicle using this method.

In certain embodiments, the user-operated control device may include a first and a second mode of operation. The method may include enabling use of either the first or second modes of operation depending on which vehicle occupant owns the hand. In this way, the same user-operated control device may be used by different vehicle occupants to control different functions and / or control systems of the vehicle. This means that the number of different user-operated control devices located in a vehicle cabin can be reduced.

The method may include controlling the use of the first mode of operation as a function of the hand associated with the first vehicle occupant and the second mode of operation as a function of the hand associated with the second vehicle occupant. This is particularly advantageous because each vehicle occupant can use the same device, for example, to influence only settings that are relevant to their own comfort.

The method may include determining whether the hand is the left or right hand of the vehicle occupant; and determining whether the hand belongs to the first vehicle occupant or to the second vehicle occupant depending on whether the hand is the left or right hand of the vehicle occupant. This provides a convenient way to identify the vehicle occupants. at For example, a driver of the vehicle may attempt to operate the user-operated control devices with his right hand in left-steering vehicles where the majority of the user-operated control devices are located in a central part of the vehicle dashboard. In contrast, in the case of a right-hand drive vehicle, the opposite is the case - the driver rather tries to operate the user-operated control devices with his left hand. Therefore, it is possible in many cases to determine to which vehicle occupant a hand belongs, depending on whether it is a right or a left hand.

The method may include: determining whether the hand is oriented within the volume of space with respect to the user-operated control device with the palm up or down; and determining whether the hand belongs to the first vehicle occupant or to the second vehicle occupant, depending on the orientation of the hand. The analysis of the orientation of the hand provides a practical way of determining whether the hand is the left or right hand of a vehicle occupant.

The method may include determining a direction of entry of the hand into the volume of space relative to the user-operated control device; and determining whether the hand belongs to the first vehicle occupant or to the second vehicle occupant depending on the direction of entry. The direction of entering the hand into the volume of space with respect to the user-operated control device provides a good indication of the location of the vehicle occupant with respect to the user-operated control device, and thus it is possible to distinguish between different vehicle occupants depending on their location relative to the user-operated control device , For example, the direction of entry of a hand belonging to a driver is from the direction in which the steering wheel is relative to the user-operated control device.

The method may include: determining the position of the hand with respect to a proximity limit associated with the user-operated control device, the proximity limit defining a limit offset from the user-operated control device by the predetermined threshold distance; and allowing use of the user-operated control device in response to the position of at least a portion of the hand that intersects the approach limit.

The method may include: obtaining image data of the hand within the volume of space; Receiving a hand-reflected reflectance signal; Determining a distance of the hand from a particular origin in dependence on the reflectance signal; and determining the position of the hand relative to the user-operated control device in dependence on the distance of the hand relative to the particular origin, the obtained image data and a known distance of the user-operated control device relative to the determined origin. The time required for the reflectance signal to be measured by a receiver is proportional to the distance of the hand from the sensor, and therefore provides a convenient way for range information associated with the position of the hand to be obtained. In this way, it is possible to determine the position of the hand with respect to the user-operated control device on the basis of a two-dimensional image of the hand related to the user-operated control device and the removal information of the hand. This greatly simplifies the hardware required to perform the method, and in particular avoids the need to use a complex system of two or more cameras, each of which is configured to acquire different perspective images of the hand related to the user-operated control device, of which the removal of the Hand based on the user-operated control device can be determined.

In certain embodiments, the particular origin may correspond to the position of an image capture device located in the vehicle cabin.

The method may include: detecting a first hand and a second hand within the volume of space, wherein the first and second hands are associated with the first and second vehicle occupants, respectively; Determining which vehicle occupant the first and second hands belong to; Determining the position of each hand with respect to the user-operated control device; and allowing use of the user-operated control device for the first vehicle occupant depending on at least a portion of the first hand being at a distance from the user-operated control that is at most equal to the predefined threshold distance. In this way, the method is advantageously capable of coping with a scenario in which two or more hands of the vehicle occupants are placed within the field of view of the camera and still capable of providing the correct functionality. This improves the versatility of the process.

The method may include preventing the use of the user-operated control device for the second vehicle occupant depending on at least a portion of the second hand being at a distance from the user-operated control device that is at most equal to the predefined threshold distance. In this way, the operation of control devices may be limited to certain vehicle occupants. For example, in certain embodiments, the first vehicle occupant may be a driver of the vehicle, the second vehicle occupant may be an occupant of the vehicle, wherein the first hand belongs to the driver of the vehicle and the second hand belongs to the occupant of the vehicle. In this example, the operation of the user-operated control device may be limited to the driver of the vehicle.

In accordance with another aspect of the present invention, a controller is provided for facilitating the use of a user-operated controller in a vehicle cabin. The controller may include an input configured to receive image data obtained by an image capture device, a processor, and an output. The processor may be configured in use for: detecting a hand of a vehicle occupant from the image, wherein the hand is located in a volume of space within the vehicle cabin within which image data is obtained by the image capture device; Determining a position of the hand with respect to the user-operated control device; and determining whether the hand belongs to a first vehicle occupant or to a second vehicle occupant. The output may be arranged in use to output a control signal that allows use of the user-operated control device in response to a distance between at least a portion of the hand and the user-operated control device corresponding to at most a predefined threshold distance, and depending on which Vehicle occupants heard the hand. The present control benefits from the same advantages as set forth with respect to the preceding aspect of the invention and its embodiments.

In certain embodiments of the controller, the input, the processor and the output may be arranged in use to provide the functionality of the method described above.

In one embodiment, the user-operated control device includes a first mode of operation and a second mode of operation, and the output is arranged to output the control signal that enables either the use of the first or second mode of operation depending on which vehicle occupant owns the hand.

The output may be configured in use to output the control signal enabling use of the first mode of operation responsive to the hand being owned by the first vehicle occupant and the second mode of operation depending on whether the hand belongs to the second vehicle occupant.

In another embodiment, the processor is disposed in use to determine if the hand is the left or right hand of the vehicle occupant; and determining whether the hand belongs to the first vehicle occupant or the second vehicle occupant, depending on whether the hand is the left or the right hand of the vehicle occupant.

In yet another embodiment, the processor is arranged in use to determine whether the hand is oriented up or down within the volume of space relative to the user-operated control device; and to determine whether the hand belongs to the first or the second vehicle occupant depending on the orientation of the hand.

Optionally, the processor is arranged in use to determine a direction of entry of the hand into the volume of space relative to the user-operated control device and to determine, depending on the direction of entry, whether the hand belongs to the first or the second vehicle occupant.

In another embodiment, the processor is arranged in use to determine the position of the hand with respect to an approach limit associated with the user-operated control device, the approach limit defining a limit offset from the user-operated control device by the predefined threshold distance; and the output in use is arranged to output the control signal enabling the use of the user-operated control device in response to the position of at least a portion of the hand that intersects the approach limit.

In yet another embodiment, the input is arranged to receive data from a time of flight ToF image capture device that includes a sensor, the ToF image capture device being arranged in use to obtain image data of the hand Hand to light and a return time of one the reflected illumination signal being proportional to the distance of the hand from the sensor, the data comprising the image data and the return time of the reflected illumination signal; and wherein the processor is disposed in use to determine the position of the hand with respect to the user-operated control device in dependence on the determined distance of the hand from the sensor, the acquired image data obtained, and a known distance of the user-operated control device relative to the sensor. wherein the distance of the hand from the sensor is determined in response to the return time of the reflected illumination signal.

In one embodiment, the processor is arranged in use to detect a first hand and a second hand within the volume of space, wherein the first and second hands belong to the first and second vehicle occupants, respectively, and to which vehicle occupant the first and second vehicle occupants second hand belongs; wherein the processor is disposed in use to determine the position of each hand with respect to the user-operated control device; and the output in use is arranged to output the control signal permitting the use of the user-operated control device in response to at least a portion of the hand of the first vehicle occupant being at a distance from the user-operated control device that corresponds at most to the predefined threshold distance.

The output may be arranged in use to output a control signal prohibiting use of the user-operated control device in response to at least a portion of the second vehicle occupant's hand being at a distance from the user-operated control device that is at most the predefined threshold distance.

In embodiments, the first vehicle occupant is a driver of the vehicle and the second vehicle occupant is an occupant of the vehicle, wherein the first hand belongs to the driver of the vehicle and the second hand belongs to the occupant of the vehicle.

Optionally, the user-operated control device comprises a display unit arranged in the vehicle cabin, and the output is arranged to output the control signal enabling the use of the display unit for the display unit.

In certain embodiments, the user-operated control device may include a display unit disposed in the vehicle cabin. The output may be arranged in use to output the control signal that allows the display unit to be used in response to a distance between at least a portion of the hand and the display unit corresponding to at most a predefined threshold distance, and depending on which Vehicle occupants heard the hand.

In accordance with yet another aspect of the invention, there is provided a vehicle configured to perform the aforementioned method.

In accordance with yet another aspect of the invention, a vehicle is provided that includes the aforementioned controller.

In accordance with yet another aspect of the invention, there is provided a computer program product comprising instructions for carrying out the aforementioned method.

The computer program product may include instructions that, when executed on a processor, configure the processor to: capture a hand of a vehicle occupant within a volume of space in the vehicle cabin; Determining a position of the hand with respect to the user-operated control device; Determining whether the hand belongs to a first vehicle occupant or to a second vehicle occupant; and allowing use of the user-operated control device in response to a distance between at least a portion of the hand and the user-operated control device corresponding to at most a predefined threshold distance and depending on which vehicle occupant the hand belongs to.

According to yet another aspect of the invention, there is provided a computer readable medium having stored thereon instructions for carrying out the aforementioned method.

The data carrier may include instructions that, when executed on a processor, configure the processor to: detect a hand of a vehicle occupant within a volume of space in the vehicle cabin; Determining a position of the hand with respect to the user-operated control device; Determining whether the hand belongs to a first vehicle occupant or to a second vehicle occupant; and allowing use of the user-operated control device in response to a distance between at least a portion of the hand and the user-operated one Control device corresponds to at most a predefined threshold distance and depending on which vehicle occupant the hand belongs.

In accordance with yet another aspect of the invention, a system is provided that includes the aforementioned controller and a ToF camera.

The controller may include an input configured to receive image data obtained by an image capture device, a processor, and an output. The processor may be configured in use to: detect a hand of a vehicle occupant from the image data, wherein the hand is within a volume of space in the vehicle cabin within which image data is obtained by the image capture device; Determining a position of the hand with respect to the user-operated control device; and determining whether the hand belongs to a first vehicle occupant or to a second vehicle occupant. The output may be arranged in use to output a control signal that allows use of the user-operated control device in response to a distance between at least a portion of the hand and the user-operated control device corresponding to at most a predefined threshold distance, and depending on which Vehicle occupants heard the hand.

Optionally, the system may include a user-operated controller.

It is expressly intended, within the scope of this application, that the various aspects, embodiments, examples and alternatives presented in the preceding paragraphs, in the claims and / or in the following description and drawings, and in particular, their individual features, be independent of each other or in any combination. This means that all embodiments and / or features of any embodiment can be combined in any manner and / or combination, provided that these features are not incompatible. The Applicant reserves the right to change any claim originally filed or to submit any new claim, including the right to alter any originally filed claim, in order to depend on and / or disregard any feature of any other claim although it was previously unclaimed in this way.

list of figures

• 1 eine schematische Darstellung eines vorderen Teils einer Fahrzeugkabine mit einer Kamera, die ein Sichtfeld aufweist, das angeordnet ist, um Bilddaten einer Hand eines Fahrzeuginsassen innerhalb eines Raumvolumens in einer Fahrzeugkabine zu erhalten;
• 2 eine Darstellung, die eine Draufsicht auf das Sichtfeld der Kamera darstellt und repräsentativ für den Teil der Fahrzeugkabine ist, der durch die Kamera aus 1 abgebildet wird;
• 3 eine schematische Darstellung der funktionalen Komponenten einer Steuerung, die konfiguriert ist, um ein Steuersignal auszugeben, um die Verwendung einer benutzerbetriebenen Steuervorrichtung zu ermöglichen, die in der Fahrzeugkabine aus 1 angeordnet ist;
• 4 ein Prozessablaufdiagramm, das ein Verfahren zum Ermöglichen der Verwendung einer benutzerbetriebenen Steuervorrichtung in einer Fahrzeugkabine in Abhängigkeit davon, zu welchem Fahrzeuginsassen eine erfasste Hand gehört, und in Abhängigkeit von der Nähe der Hand eines Fahrzeuginsassen zu der Steuervorrichtung unter Verwendung der Kamera aus 1 erläutert;
• 5 eine schematische Darstellung, die das Funktionsprinzip einer Flugzeit(ToF)kamera hervorhebt, die verwendet werden kann, um die Position der Hand eines Fahrzeuginsassen in der Fahrzeugkabine aus 1 zu bestimmen;
• 6a und 6b schematische Darstellungen, die eine dreidimensionale Punktewolke einer Hand eines Fahrzeuginsassen zeigen, die unter Verwendung der ToF-Kamera aus 5 erzeugt wurde;
• 7 eine schematische Darstellung eines Fahrzeugs, das die Kamera aus 1 und die Steuerung aus 3 umfasst.

One or more embodiments of the invention will now be described by way of example only with reference to the accompanying drawings, in which:

• 1 a schematic representation of a front part of a vehicle cabin with a camera having a field of view, which is arranged to obtain image data of a hand of a vehicle occupant within a volume of space in a vehicle cabin;
• 2 a representation, which is a plan view of the field of view of the camera and is representative of the part of the vehicle cabin, through the camera 1 is shown;
• 3 a schematic representation of the functional components of a controller which is configured to output a control signal to enable the use of a user-operated control device in the vehicle cabin 1 is arranged;
• 4 3 is a process flow diagram illustrating a method for enabling the use of a user-operated control device in a vehicle cabin depending on which vehicle occupant a detected hand belongs to and the proximity of the hand of a vehicle occupant to the control device using the camera 1 explains;
• 5 a schematic representation that highlights the principle of operation of a flight time (ToF) camera, which can be used to the position of the hand of a vehicle occupant in the vehicle cabin 1 to determine;
• 6a and 6b schematic representations showing a three-dimensional point cloud of a hand of a vehicle occupant, using the ToF camera off 5 was generated;
• 7 a schematic representation of a vehicle, the camera off 1 and the controller off 3 includes.

DETAILED DESCRIPTION

1 is a perspective view of a part of the vehicle cabin 1 and in particular shows the driver 3 in the driver's seat 5 sitting. An image capture device in the form of a camera 7 that a field of vision 9 that is in 1 through lines 11 is sketched, is shown arranged in the cabin roof. Optionally, the camera 7 include a flight time (ToF) camera. The camera 7 is arranged so that it depicts objects that are within the field of view 9 the camera. The Field of view defines a volume of space within the vehicle cabin within which objects pass through the camera 7 be imaged. The camera 7 is arranged so that a dashboard 13 in the vehicle cabin 1 within the field of view 9 the camera is lying. The dashboard 13 includes a plurality of different user-operated control devices 15 which may relate to, but are not limited to: air ventilation switches; C switch; Vehicle infotainment system; Vehicle display units; Recirculation switch; and any other control device configured to operate a control system of the vehicle. The vehicle cabin 1 can be in an in 7 illustrated vehicle 43 be included.

In certain embodiments, the camera 7 operational with a controller 19 (in 3 shown) configured to receive image data from the camera 7 and a control signal to a specific user-operated control device 15 in response to an analysis of the received image data. The control 19 can be configured to use a specific user-driven control device 15 is enabled if at least a part of the hand of a vehicle occupant 17 is arranged at a distance which is at most a predefined threshold distance from the specific user-operated control device 15 corresponds to which vehicle occupant the hand belongs to. For example, a first vehicle occupant and a second vehicle occupant may be in the vehicle cabin, and the use of a user-operated control device may be limited to either the first or the second vehicle occupant.

For example, the first vehicle occupant may be a driver of the vehicle, and the second vehicle occupant may be an occupant of the vehicle. The camera 7 in combination with the controller 19 Can be used to hand 17 identify a vehicle occupant and determine whether the hand belongs to the driver or the occupant of the vehicle. The use of the user-operated control device 15 can then be limited to the desired vehicle occupants.

2 shows a perspective view of the dashboard 13 within the field of view 9 the camera, which has a plurality of user-operated control devices 15 includes. The analysis of the camera 7 taken pictures taken by the controller 19 be executed, it allows the controller 19 to determine which vehicle occupant owns a hand within the field of view 9 the camera is arranged. To identify an image of a hand, the controller can 19 and especially the processor 25 configured with an image recognition software configured to be a hand of a vehicle occupant 17 that are within the field of view 9 the camera is arranged to identify from received image data. The hand of the vehicle occupant may be identified in various ways to belong to a first vehicle occupant or a second vehicle occupant, as further explained in the following description.

In certain embodiments, the field of view may 9 The camera can be divided into two sections, a left enclosure zone 14 and a right confinement zone 16 , The hand of a vehicle occupant 17 is shown as being both in the left confinement zone 14 as well as in the right confinement zone 16 entry. The containment zones may be used to determine whether the hand belongs to a first vehicle occupant or to a second vehicle occupant. This can be achieved, for example, by determining whether the hand first enters the left confinement zone 14 or in the right confinement zone 16 entry. This will be explained in more detail in the following description.

As mentioned earlier, the camera can 7 in certain embodiments, operatively with a controller 19 (in 3 shown). The controller may be configured to display image data from the camera 7 receive and receive a control signal to a user-operated control device 15 depending on an analysis of the received image data.

3 provides a functional overview of the controller 19 ready. The control 19 Can be functional in an existing electronic control unit of the vehicle 43 be embedded. The control 19 can with an entrance 21 and an exit 23 be provided. The entrance 21 can be configured to receive image data from the camera 7 be obtained while the output 23 may be configured to provide a control signal to a specific user-operated control device 15 issue. The control 19 can be a processor 25 which is configured to: Analyze the from the camera 7 received image data, identifying image objects such as the hand 17 of the vehicle occupant in the obtained image data, analyzing the image of the hand of the vehicle occupant to determine to which vehicle occupant the hand belongs and to generate control signals enabling a particular user-operated control device in dependence on the relative position of the hand of the vehicle occupant 17 to the particular user-operated control device 15 and depending on it, too which vehicle occupant owns the hand.

In use, image data are taken by the hand of a vehicle occupant through the camera 7 get it, and it gets through the control 19 determined, typically by the processor 25 the controller 19 to which user occupant owns the hand, and their position is determined relative to the specific user-operated control device, so that the use of the specific user-operated control device via a control signal supplied by the controller 19 is issued, can be enabled.

In certain embodiments, the controller may 19 be configured in use to control the control signal in dependence on a distance between at least part of the hand 17 a vehicle occupant and the user-operated control device 15 output that corresponds at most to a predefined threshold distance, and also depending on which vehicle occupant's hand 17 belongs. For example, if the camera 7 Image data of a hand 17 a vehicle occupant, such. As the driver's hand receives, the controller can 19 configured to identify the image of the hand within the received image data received. The corresponding position of the imaged hand with respect to the user-operated control device can then be determined from the obtained image data. The control 19 may also be configured to identify which vehicle occupant owns the hand.

4 FIG. 11 is a process flow diagram outlining the method according to certain embodiments of the invention to illustrate the use of a user-operated control device 15 depending on the proximity of the hand 17 a vehicle occupant to the user-operated control device and depending on which vehicle occupant's hand 17 belongs, to allow.

The procedure is done by the camera 7 initiated, the image data in the vehicle cabin 1 in step 301 receives. In certain embodiments, the camera 7 be configured to obtain the image data continuously or periodically to obtain image data having a predefined frequency. The obtained image data can be sent to the controller 19 be forwarded to analysis where in step 303 obtained image data can be analyzed to a hand 17 of a vehicle occupant within the obtained image data. As previously mentioned, this may include the use of image recognition software. Once the hand 17 a vehicle occupant has been identified within the obtained image data is in step 305 determines to which vehicle occupant the hand belongs. This can be done by the processor 25 by analyzing the obtained image data to determine which vehicle occupant owns the hand. In particular, it is determined whether the hand belongs to a vehicle occupant who is authorized to use the associated user-operated control device. If it is determined that the hand belongs to a vehicle occupant who is not authorized, the method returns to step 303 back. If, instead, the hand belongs to a vehicle occupant who is authorized, the user-operated control device in question 15 to use, the method goes to step 306 continued. In step 306 becomes the position of the hand 17 relative to the user-operated control device 15 certainly. The position of the hand 17 relative to the user-operated control device 15 can through the processor 25 be determined. When the vehicle dashboard 13 a plurality of different user-operated control devices 15 includes, can step 306 determining the position of the hand 17 with respect to the next user-operated control device 15 include. In step 307 It determines if at least part of the hand 17 is at a distance which is at most a predefined threshold distance from the user-operated control device 15 equivalent. If it is determined that no part of the hand is within the predefined threshold distance, then the processor is running 25 to analyze received image data, and the process returns to step 303 back. If instead by the processor 25 determining that at least a portion of the identified hand is within the predefined threshold distance of the user-operated control device 15 The processor then generates a control signal for outputting to the relevant user-operated control device 15 , in step 308 , Upon receipt of the control signal, step 310 allows the use of the user-operated control device. In this way, the use of user-operated control devices can be selectively enabled for specific vehicle occupants. For example, in this way, a driver of the vehicle may be allowed to use specific user-operated control devices whose use is not made possible for other vehicle occupants, for example for an occupant.

In certain embodiments, the step 305 the determination of whether the hand belongs to an authorized vehicle occupant after determining whether at least a portion of the hand is at a distance equal to or less than the predefined threshold distance: in other words, after the step 307 was executed. The exact order in which the steps 305 and 307 be executed, is irrelevant for the present purposes.

In certain embodiments, a user-operated controller may include a first mode of operation and a second mode of operation, each of which upon receipt of a control signal in step 310 depending on which vehicle occupant's hand belongs to the controller 19 within the predetermined threshold distance of the user-operated control device 15 has detected, can be activated.

In certain embodiments, a first mode of operation may be activated in response to the hand being associated with the first vehicle occupant, and a second mode of operation may be activated in response to the hand being associated with the second vehicle occupant. For example, a climate control device configured to control an ambient temperature within the vehicle cabin may include a first operating mode for controlling the temperature of the driver side of the vehicle and a second operating mode for controlling the temperature of a passenger side of the vehicle. Depending on whether by the controller 19 detected hand as belonging to the driver or to the occupant, the first and the second operating mode can be activated.

As mentioned, through an analysis of image data taken by the camera 7 be determined, to which vehicle occupants a hand belongs.

In certain embodiments, it may be determined from obtained image data whether the hand is the left hand or the right hand of the vehicle occupant. For example, this can be determined by analyzing the position of the fingers relative to the thumb of the hand. It may then be determined whether the hand belongs to the first vehicle occupant or the second vehicle occupant, depending on whether the hand is the left or the right hand of the vehicle occupant. Since most user-operated control devices are located within the central section of the dashboard 13 may be disposed within the vehicle cabin, for example, a first vehicle occupant and a second vehicle occupant on each side of the user-operated control devices 15 sit. When a first vehicle occupant with respect to the field of view of the camera on the left side of the user-operated control device 15 sitting, it is very likely that he uses his right hand to the control devices 15 to operate. Therefore, if a right hand from the camera 7 is detected, it can be assumed that this hand belongs to the first vehicle occupant in this situation. When a second vehicle occupant is seated on the right side of the user-operated control device and a left hand is detected by the camera, it can be assumed that this hand belongs to the second vehicle occupant.

Further, the vehicle occupant may have his hand oriented with the palm up or down within the volume of space, within the field of view 9 the camera is received, based on the user-operated control device 15 ,

In certain embodiments, by the processor 25 be determined by analyzing the light absorption by the hand of the vehicle occupant, whether the hand of the vehicle occupant with the palm up or down with respect to the user-operated control device 15 is oriented. The skin texture of a palm differs from the skin texture of the back of the hand, and as a result, the amount of incident light absorbed by the palm varies from the amount of incident light absorbed by the back of the hand. Accordingly, it is by configuring the controller 19 For analyzing the intensity of the reflected signal indicative of the amount of incident light absorbed by the hand, it is possible for the controller 19 determines whether the hand is oriented with the palm up or down. By determining whether the vehicle occupant is holding his palm up or down within the volume of the room relative to the user-operated control device 15 Therefore, it can be determined with greater accuracy whether the hand is a left hand or a right hand. It can therefore be determined in the same way as described above whether the hand belongs to a first vehicle occupant or a second vehicle occupant.

In certain embodiments, the controller may also determine whether the hand of a vehicle occupant is from the left or right side of the user-operated control device 15 in the field of view 9 the camera enters. As in 2 shown, the field of view 9 the camera into a left confinement zone 14 and a right confinement zone 16 be split. When a hand is detected entering the left enclosure zone 14 The controller may determine that the hand enters the field of view of the camera from the left and vice versa for a hand that enters the right confinement zone 16 entry.

Depending on the direction of hand entry, it can be determined whether the hand belongs to a first vehicle occupant or a second vehicle occupant. For example, in the situation where user-operated control devices 15 inside a central dashboard 13 of vehicle 43 are arranged when a vehicle occupant left of this central dashboard 13 Sitting, his hand most likely from the left in the field of view 9 penetrate the camera, or into the left confinement zone 14 enter. Similarly, it may be determined whether a hand detected by the controller enters the right confinement zone 16 To enter, belong to a vehicle occupant, on the right side of the central dashboard 13 and user-operated control devices 15 sitting.

Depending on the country of manufacture of the vehicle and / or the country in which the vehicle is most likely to be used, the driver of the vehicle may turn left or right of the central dashboard 13 that's the user-operated control devices 15 contains, and an occupant on the opposite side of the central dashboard 13 sit. It can therefore be determined whether the hand of a vehicle occupant belongs to a driver or an occupant.

In certain embodiments, the predefined threshold distance may be between at least a portion of the hand 17 a vehicle occupant and the user-operated control device 15 to a few centimeters, for example a distance within the range of 1 cm to 10 cm, including 1 cm and 10 cm. In certain embodiments, the predefined threshold may be a control proximity limit that the user controller 15 outline and offset by the predetermined threshold distance therefrom, wherein the control proximity limit, when crossed by at least a portion of the hand of the vehicle occupant, controls 19 causing the control signal to be output to the user-operated control device concerned 15 to create.

The control proximity limit may be geometrically shaped. For example, the control proximity limit may be box shaped or spherically shaped. Effectively, the control proximity limit may be adapted to one of the user-operated controller 15 refer to the predefined threshold distance offset volume. In response to a portion of the control proximity limit being crossed by at least a portion of the vehicle occupant's hand, the controller generates the control signal for output to the user-operated controller 15 , It should be noted that not all parts of the control proximity limit from the user-operated control device 15 have to be offset by the predefined threshold distance. For example, if the control proximity limit is box-shaped (eg, cube-shaped), it should be noted that some areas of the cube may not be offset by the predefined threshold distance from the user-operated controller.

To allow the position of the hand 17 relative to the user-operated control device 15 is determined, the camera can 7 in certain embodiments relate to a 3D mapping control arranged to provide a 3D model of the hand within the field of view 9 to create. For example, the camera may be 7 in certain embodiments relate to a time of flight (ToF) camera in which each detected image pixel is connected to a distance based on a return time of a reflected illumination signal. To accomplish this, the ToF camera may be configured with an illumination source arranged to illuminate the field of view of the camera. The incident illumination signal is then reflected by objects present in the field of view of the camera, and the return time of the reflected illumination signal is measured. In this way it is possible to assign a distance measurement to each imaged object. The illumination signal may refer to any electromagnetic signal and need not be included in the visible spectrum. For example, in certain embodiments, the illumination signal may be operated in the infrared spectrum.

In those embodiments in which the camera 7 includes a ToF camera, the controller can 19 and in particular the entrance 21 be configured to receive both camera image data and image removal information from the ToF camera. This allows the controller 19 and especially the processor 25 , the position of the hand 17 the vehicle occupant relative to the user-operated control device 15 from the received data.

5 is a schematic diagram illustrating the operation principle of a ToF camera 27. A modulated illumination source 29 is used to a desired destination 31 to illuminate. The incident lighting 33 gets through the goal 31 reflected and on a sensor 35 detected comprising an array of pixels. However, while at the same time the reflected modulated light 37 is detected, capture the pixels of the sensor 35 also visible light reflected from the target. Because the lighting signal 33 It can be modulated by the visible light coming from the target 31 is reflected, which makes it possible to measure the time of flight of the modulated illumination signal. The flight time that the modulated lighting signal needed to get to the destination 31 to meet and to the sensor 35 Being reflected back is measured when it is on the sensor 35 falls. In this way, each detected image pixel can be removed corresponding image object on the basis of the measured flight time assigned to that of the sensor 35 reflected modulated illumination signal to be measured 37 is required. More specific details regarding the operation of ToF cameras are widely used in the art, and for this reason a more detailed discussion is not necessary for the present purposes.

If the camera 7 out 1 a ToF camera 27 it is possible to generate a three-dimensional point cloud of the hand of the vehicle occupant, which is within the field of view 9 the camera is located. The 6a and 6b represent an example of a three-dimensional point cloud 39 the hand 17 of the vehicle occupant using the ToF camera 27 is produced. In certain embodiments, the controller may 19 be configured to use the three-dimensional point cloud using the ToF camera 27 received image data and image removal information. 6a shows a point cloud 39 the hand 17 of the vehicle occupant when approaching a rectangular control proximity boundary 41 approaches. In 6b cuts a part of the point cloud 39 the hand 17 of the vehicle occupant a part of the control proximity limit 41 , In this case and as previously mentioned, the controller can 19 depending on which vehicle occupant owns the hand, configured to provide a control signal for output to the user-operated control device 15 to enable the use of this user-operated control device.

To allow the position of the hand of the vehicle occupant 17 with respect to a user-operated control device 15 is determined, the position of the user-operated control device 15 determined based on the ToF camera. This can in turn be done using an image recognition software. Since the position of the user-operated control device 15 related to the ToF camera 27 is known and the position of the hand of the vehicle occupant 17 related to the ToF camera 27 is known, the position of the hand of the vehicle occupant 17 with respect to the user-operated control device 15 be determined using basic trigonometry. In certain embodiments, and to facilitate calculation during use, the controller may 19 during the initial configuration of the ToF camera 27 with distance information from each user-operated control device 15 related to the ToF camera 27 be provided. This distance information can be stored and retrieved for subsequent use when needed. This facilitates subsequent calculation of the position of the hand relative to a desired user-operated control device 15 because only the removal of the hand of the vehicle occupant 17 in terms of the ToF camera 27 and the position relative to the known position of the user-operated control device 15 requires a calculation, both of which can be obtained from data obtained from the ToF camera 27 be recorded.

While the foregoing embodiments of the invention have been described in the context of a ToF camera, it should be understood that alternative camera configurations may be used in accordance with embodiments heretofore described. Any configuration of cameras may be used that allows one to capture image data of a hand relative to the particular surface area and determine the position of the hand relative to the particular surface area. For example, a configuration of two or more cameras, each configured to provide a different perspective image of the hand with respect to the surface area to be detected, may also be used. In such an arrangement, the various perspective images of the hand relative to the particular surface area would allow the controller to determine the position of the hand with respect to the particular surface area by triangulation.

Similarly, in an alternative embodiment, the ToF camera of the previous embodiments may be replaced by a conventional camera in combination with an optical ruler, such as a LIDAR. In such an embodiment, the lidar provides the image object removal information while the camera provides image data. The controller, in such embodiments, may be configured to analyze the LIDAR data in combination with the obtained image data to determine the position of the hand of the vehicle occupant with respect to the determined surface area.

In certain embodiments, the hand of a vehicle occupant may be within the field of view 9 the camera are lying. For example, both the first and second vehicle occupants may desire to control either different user-operated control devices or the same user-operated control device at the same time. When a first hand associated with a first vehicle occupant and a second hand associated with a second vehicle occupant both simultaneously in the field of view 9 the camera can be detected, the processor 25 be able to identify which hand belongs to which vehicle occupant. The processor 25 Then, the position of each hand in relation to a user-driven Determine control device. The use of the user-operated control device 15 may be activated for the first vehicle occupant depending on at least a portion of the first hand being within the threshold distance of the user-operated control device.

In other embodiments, the controller 19 in addition to allowing the use of a user-operated control device 15 for the first user additionally prevent the second vehicle occupant from the user-operated control device 15 in response to the second hand being within the threshold distance of the user-operated control device 15 lies, used. In this way, the use of a particular user-operated control device 15 only to a first vehicle occupant, even in the scenario where the hand of a second vehicle occupant is also within the approximate limit of the user-operated control device 15 is available. The second vehicle occupant can therefore be prevented from using a user-operated control device 15 which he is not entitled to use, even if the hand of a first vehicle occupant is present, making the system more robust.

For example, in an embodiment where the first vehicle occupant is the driver of the vehicle and the second vehicle occupant is an occupant of the vehicle, it may be desirable for only the driver of a vehicle to operate a controller that controls the driving mode of the vehicle. If a passenger changes the driving mode of the vehicle while the driver is driving the vehicle, the safety may be impaired. Therefore, it may be advantageous for the occupant to be prevented from using this particular user-operated control device, including when the driver's hand is in the vicinity of the user-operated control device. This functionality may be particularly useful in the situation where the occupant is a child who may change important vehicle settings while playing with the user controls.

In other embodiments, the use of a first user-operated control device 15 for the first vehicle occupant, and the use of a second user-operated control device 15 may be enabled for the second vehicle occupant in the situation where both a first and a second hand are within the approximate limits for the first and second user-operated control devices, respectively 15 be recorded.

In certain embodiments, the user-operated control device 15 include a display unit located in the vehicle cabin. The display unit may include a user interface that may enable a vehicle occupant to change multiple vehicle settings, for example, via a touch screen or various keys. In one example, the use of the entire display unit may be enabled in response to a hand being detected within the proximity of the display unit and depending on which vehicle occupant the hand belongs to. In other embodiments, certain functionalities of the display unit may be activated for different users depending on whether their hand is within the proximity limit. In one example, an occupant of the vehicle may be enabled to operate the navigation system of the vehicle via a touch screen, but it may not be possible to change the driving mode of the vehicle via the same touch screen.

In some embodiments, the first vehicle occupant may instead be an occupant of the vehicle, and the second vehicle occupant may be the driver of the vehicle. In other embodiments, for example in the case of an autonomous passenger car, both the first and second vehicle occupants may be occupants of the vehicle.

In other embodiments, it may be possible to accommodate a greater number of vehicle occupants, such as the vehicle occupants seated in the rear seats of the vehicle 43 are located. For example, it may have additional inclusion zones 14 . 16 within the field of view 9 the camera to accommodate a larger number of vehicle occupants. In one example, there may be a third inclusion zone from the viewpoint of the camera 7 is disposed below the left and right confinement zones. The processor 25 can then determine whether the hand of a vehicle occupant of one of the rear seats within the vehicle 43 in the field of view of the camera 9 occurred. In this situation, the controller can 19 be configured to prevent the use of a user-operated control device for these vehicle occupants. For example, children may sit in the back seats of the vehicle, and it may be preferable if they can not change vehicle settings. In other embodiments, vehicle occupants in the rear seats may be permitted certain user-operated control devices 15 to use.

It should be noted that many modifications can be made to the above examples and embodiments without departing from the scope of the present invention as defined in the appended claims.

Claims (12)

A method for enabling the use of a user-operated control device in a vehicle cabin, the method comprising: Detecting a hand of a vehicle occupant within a volume of space in the vehicle cabin; Obtaining image data of the hand within the volume of space; Receiving a hand-reflected reflectance signal; Determining a distance of the hand from a particular origin in dependence on the reflectance signal; Determining a position of the hand with respect to the user-operated control device in dependence on the distance of the hand relative to the particular origin, the obtained image data, and a known distance of the user-operated control device relative to the determined origin; Determining whether the hand belongs to a first or second vehicle occupant; and Enabling use of the user-operated control device in response to a distance between at least a portion of the hand and the user-operated control device corresponding to at most a predefined threshold distance and depending on which vehicle occupant the hand belongs to. Method according to Claim 1 wherein the user-operated control device comprises a first mode of operation and a second mode of operation, and the method comprises: enabling use of the first or second mode of operation depending on which vehicle occupant the hand belongs to; the method optionally comprising: enabling use of the first mode of operation in response to the hand belonging to the first vehicle occupant and the second mode of operation depending on the hand being associated with the second vehicle occupant. Method according to one of the preceding claims, comprising: Determining whether the hand is oriented within the volume of space with respect to the user-operated control device with the palm up or down; and Determining whether the hand belongs to the first or second vehicle occupant depending on the orientation of the hand; and / or the method comprises: Determining a direction of entry of the hand into the volume of the room relative to the user-operated control device; and Determining whether the hand belongs to the first or the second vehicle occupant, depending on the direction of entry. A method according to any one of the preceding claims, wherein the particular origin coincides with a position of an image capture device. Method according to one of the preceding claims, comprising: Detecting a first and a second hand within the volume of space, wherein the first and second hands are associated with the first and second vehicle occupants, respectively; Determining which vehicle occupant the first and second hands belong to; Determining the position of each hand with respect to the user-operated control device; and Enabling the use of the user-operated control device for the first vehicle occupant in response to at least a portion of the first hand being at a distance from the user-operated control device that is at most equal to the predefined threshold distance; the method optionally comprising: Preventing the use of the user-operated control device for the second vehicle occupant in response to at least a portion of the second hand being at a distance from the user-operated control device that corresponds at most to the predefined threshold distance; wherein the first vehicle occupant is a driver of the vehicle, the second vehicle occupant is an occupant of the vehicle, the first hand belongs to the driver of the vehicle and the second hand belongs to the occupant of the vehicle. A controller for enabling the use of a user-operated control device in a vehicle cabin, comprising: an input configured to receive image data obtained by a time-of-flight (TOF) image sensing device including a sensor, wherein the ToF image capture device is arranged to receive image data of the hand, to illuminate the hand and to measure the return time of a reflected illumination signal, wherein the return time of the reflected illumination signal is proportional to the distance of the hand from the sensor, the data being the image data and the return time of the reflected illumination signal; a processor configured for: Recognizing a hand of a vehicle occupant from the image data, wherein the hand is within a volume of space in the vehicle cabin within which image data is obtained by the image capture device; Determining a position of the hand with respect to the user-operated control device in dependence on the removal of the hand from the sensor, the image data obtained and a known distance of the user-operated control device relative to the sensor, the distance of the hand from the sensor being dependent on the return time of the sensor reflected illumination signal is determined; Determining whether the hand belongs to a first or second vehicle occupant; and an output, disposed in use, for outputting a control signal that allows use of the user-operated control device in response to and depending on a distance between at least a portion of the hand and the user-operated control device corresponding to a predefined threshold distance which vehicle occupant owns the hand. Control after Claim 6 wherein the user-operated control device comprises a first operating mode and a second operating mode and the output is arranged to output the control signal allowing the use of either the first or the second operating mode depending on which vehicle occupant the hand belongs to Output in use is configured to output the control signal enabling the use of the first mode of operation in response to the hand being associated with the first vehicle occupant and the second mode of operation depending on the hand being associated with the second vehicle occupant; is possible. Control according to one of the Claims 6 or 7 wherein the processor is disposed in use to determine whether the hand is oriented within the volume of space with respect to the user-operated control device with the palm up or down; and determining whether the hand belongs to the first or second vehicle occupant depending on the orientation of the hand; and / or wherein the processor is disposed in use to determine a direction of entry of the hand into the volume of space relative to the user-operated control device and in response from the direction of entry to determine whether the hand belongs to the first or the second vehicle occupant. Control according to one of the Claims 6 to 8th wherein the processor is disposed in use to detect a first and a second hand within the volume of space, wherein the first and second hands are associated with the first and second vehicle occupants, and to which vehicle occupant the first and second occupants Hand belong; the processor in use is arranged to determine the position of each hand with respect to the user-operated control device; and the output in use is arranged to output the control signal that allows the user-operated control device to use at least a portion of the hand of the first vehicle occupant at a distance from the user-operated control device that is at most equal to the predefined threshold distance: optional wherein the output is in use to output a control signal prohibiting use of the user-operated control device in response to at least a portion of the second vehicle occupant's hand being at a distance from the user-operated control device that is at most equal to the predefined threshold distance: and optionally wherein the first vehicle occupant is a driver of the vehicle and the second vehicle occupant is an occupant of the vehicle, wherein the first hand belongs to the driver of the vehicle and the second hand belongs to the occupant of the vehicle Vehicle belongs. Control according to one of the Claims 6 to 9 wherein the user-operated control device comprises a display unit arranged in the vehicle cabin, and the output is disposed in use to output the control signal enabling the use of the display unit for the display unit. Vehicle that is arranged in use to follow the procedure of one of Claims 1 to 5 execute and / or control according to one of Claims 6 to 10 includes. Computer program product, instructions for performing the method according to one of Claims 1 to 5 comprising and / or computer-readable medium on which instructions for carrying out the method according to one of Claims 1 to 5 are stored.

Images