GB2574877A

GB2574877A - Apparatus and methods for controlling a transition between driving modes

Info

Publication number: GB2574877A
Application number: GB201810240A
Authority: GB
Inventors: Gemma Matthias Rebecca; Skrypchuk Lee; Lavie Nilli; Brighton James
Original assignee: Cranfield University; Jaguar Land Rover Ltd; UCL Business Ltd
Current assignee: Cranfield University; Jaguar Land Rover Ltd; UCL Business Ltd
Priority date: 2018-06-22
Filing date: 2018-06-22
Publication date: 2019-12-25
Anticipated expiration: 2038-06-22
Also published as: DE102019208732A1; GB201810240D0; GB2574877B

Abstract

A control system for controlling a transition between an autonomous driving mode and a manual driving mode. The control system may comprise one or more controllers, e.g. steering controller 110, configured to receive a condition signal indicative of a driver’s 10 condition and determine a haptic feedback signal (210, fig 2) indicative of haptic feedback to be applied to a control, e.g. steering wheel 120, of a vehicle. The haptic feedback signal (210) is determined in dependence on the condition signal and an expected or desired driver control input 25 for driving the vehicle in the manual mode. The control system outputs the haptic feedback signal to cause application of haptic feedback to the control, i.e. steering wheel 120. The condition of the driver may be one or both of a mental or physical condition, e.g. muscle readiness, of the driver 10. Reference is also made to a system, a vehicle and a method of a transition between an autonomous driving mode and a manual driving mode.

Description

APPARATUS AND METHODS FOR CONTROLLING A TRANSITION BETWEEN DRIVING MODES

TECHNICAL FIELD

The present disclosure relates to controlling a transition between driving modes of a vehicle and particularly, but not exclusively, to controlling a transition between an autonomous driving mode and a manual driving mode of a vehicle. Aspects of the invention relate to a control system, to a system, and to a vehicle, to a method and to computer software.

BACKGROUND

Increasingly vehicles are being provided with an autonomous driving capability, where the motion of the vehicle in one or both of a lateral and longitudinal direction may be controlled by one or more vehicles systems, without substantive input from a driver I occupant of the vehicle. For some levels of automation, it is required that a driver of the vehicle be able to resume manual control of the vehicle in certain circumstances such as when requested by an autonomous driving system of the vehicle. Since in some instances the driver may be free to engage in non-driving tasks, such as for example reading, whilst the vehicle is being driven in certain autonomous modes, the driver is unlikely to be able to instantaneously resume control of the vehicle. Difficulties may arise in a period of time between the driver being requested to resume manual control of the vehicle and the driver being ready to manually control the vehicle.

It is an object of embodiments of the invention to at least mitigate one or more of the problems of the prior art.

SUMMARY OF THE INVENTION

Aspects and embodiments of the invention provide a control system, a system, a method and computer software as claimed in the appended claims.

According to an aspect of the invention, there is provided a control system for controlling a transition between an autonomous driving mode and a manual driving mode, the control system comprising one or more controllers, configured to: determine a haptic feedback signal indicative of haptic feedback to be applied to a control of the vehicle, wherein the haptic feedback signal is determined in dependence on a condition signal indicative of a condition of an occupant of the vehicle, and a desired driver control input for driving the vehicle in the manual mode. Advantageously the haptic feedback is determined in dependence on the condition of the driver to aid the driver in assuming control of the vehicle.

When used herein and throughout the specification, an “autonomous mode” is intended to cover driving modes wherein the motion of the vehicle in one or both of a lateral and longitudinal direction is controlled by one or more vehicles systems, without substantive input from a driver I occupant of the vehicle. This is intended to cover systems whereby the driver I occupant is still responsible for controlling the motion of the vehicle in a first direction (e.g. in a longitudinal direction) but where one or more vehicle systems control the motion of the vehicle in a second direction (e.g. in a lateral direction). Equally, an “autonomous” mode covers modes wherein one or more vehicles are in complete control of the vehicle’s motion and, when operational, do not require an input from a driver I occupant of the vehicle. A “manual” driving mode is to be read a mode whereby the driver is responsible for controlling motion of the vehicle.

The control system may comprise an input means for receiving a transition signal indicative of a request to transition between the autonomous driving mode and the manual driving mode. Advantageously the control system may determine the haptic feedback in dependence on a start of transition between the autonomous driving mode and the manual driving mode. The transition signal may be provided in dependence on an output of a controller or a user input.

According to an aspect of the present invention, there is provided a control system for controlling a transition between an autonomous driving mode and a manual driving mode, the control system comprising one or more controllers configured to: receive a transition signal indicative of a request to transition between the autonomous driving mode and the manual driving mode and a condition signal indicative of a condition of a driver of the vehicle, determine a torque control signal indicative of an amount of torque to be applied to a control of the vehicle, wherein the torque control signal is determined in dependence on the condition signal and a desired driver control input for driving the vehicle in the manual mode, and output the torque control signal. Advantageously the amount of torque is determined in dependence on the condition of the driver to aid the driver in assuming control of the vehicle.

According to an aspect of the present invention, there is provided a control system for controlling a transition between an autonomous driving mode and a manual driving mode, the control system comprising one or more controllers, configured to: receive a condition signal indicative of a condition of a driver of the vehicle, determine a haptic feedback signal indicative of haptic feedback to be applied to a control of the vehicle, wherein the haptic feedback signal is determined in dependence on the condition signal and a desired driver control input for driving the vehicle in the manual mode, and output the haptic feedback signal to cause an application of haptic feedback to a control of the vehicle. Advantageously the haptic feedback is determined in dependence on the condition of the driver to aid the driver in assuming control of the vehicle.

In embodiments, the one or more controllers may collectively comprise: at least on electronic processor having an electrical input for receiving the condition signal; and at least one memory device coupled to the at least one electronic processor and having instructions stored therein; wherein the at least one electronic processor is configured to access the at least one memory device and execute the instructions stored therein so as to determine the haptic feedback signal.

The driver may be a primary occupant of the vehicle, wherein the primary occupant is not necessarily always required to drive the vehicle but may be a person within the vehicle responsible for driving the vehicle when in the manual mode.

The desired driver control input may be an expected input from the driver for controlling the vehicle in the manual mode. The expected input may correspond to an input provided from an autonomous driving system of the vehicle. The expected input may be that which would be expected from the driver to navigate the vehicle in a current environment of the vehicle. The expected input may be determined based upon a model of the vehicle.

The condition signal may be indicative of an estimate of a cognitive condition of the driver of the vehicle. Advantageously the condition signal may indicate the driver’s mental readiness to assume control of the vehicle.

The cognitive condition is optionally a cognitive workload of the driver of the vehicle. Advantageously the cognitive workload may indicate an ability of the driver to assume control of the vehicle, where their ability is impacted by a requirement to deal with other information or tasks.

The condition signal may be indicative of an estimate of muscle readiness of the driver of the vehicle to assume control of the vehicle in the manual driving mode. Advantageously the condition signal may indicate the driver’s physical readiness to assume control of the vehicle

The control system may be arranged to determine the haptic feedback signal having a magnitude determined in dependence on the condition signal and a direction determined in dependence on desired driver control input. Advantageously the haptic feedback may encourage the driver to provide an input in the direction of the desired driver control input. Advantageously the magnitude may prompt the driver to increase their input.

The control system may be arranged to determine a desired control authority allocated to the driver of the vehicle in dependence on the condition signal. Advantageously the control system is configured to determine the desired control authority to reflect the driver’s state.

The control system may be arranged to determine an error indicative of a difference between the desired control authority and a current degree of control authority of the driver of the vehicle, and to determine the haptic feedback signal in dependence thereon. Advantageously the control system is configured to determine the haptic feedback signal in dependence on the degree to which the current control authority of the driver is inconsistent with the desired control authority. In some embodiments, the difference may indicate a degree to which the current degree of control authority is below the desired control authority.

The control system may optionally be configured to receive an actual control input signal indicative of an actual control input of the driver of the vehicle. Advantageously the control system is provided with an indication of the driver’s input to controlling the vehicle.

The control system may optionally be configured to determine the current degree of control authority in dependence on the desired control input for the vehicle and the actual control input of the driver of the vehicle. Advantageously the current degree of control authority is based on the driver’s actual input to controlling the vehicle.

The control system may be configured to determine a first part of the haptic feedback signal in dependence on the condition signal. The first part of the haptic feedback may advantageously be indicative of the condition of the driver.

The control system may optionally be configured to receive an actual control input signal indicative of an actual control input of the driver of the vehicle. Advantageously the control system is provided with information about the driver’s control inputs to the vehicle.

Optionally the control system is arranged to determine a control difference between the desired driver control input for the vehicle and the actual control input of the driver of the vehicle, and to determine the haptic feedback signal in dependence thereon.

The control system may be configured to determine a second part of the haptic feedback signal in dependence on the control difference. Advantageously the second part of the haptic feedback reflects a differential between the driver’s actual and desired inputs.

The control system may be configured to determine the first part of the haptic feedback signal as a feedback component of the haptic feedback signal and the second part of the haptic feedback signal as a feedforward component of the haptic feedback signal. Advantageously an efficient control scheme is used to determine the haptic feedback.

According to an aspect of the present invention, there is provided a system, comprising the control system as described above, and condition sensing means for determining a condition of a driver of the vehicle and providing the condition signal to the control system in dependence thereon.

The condition sensing means may comprise an imaging device for providing image data of the driver. The condition sensing means may comprise a temperature monitoring device for determining a temperature of at least a portion of the driver’s body. The condition sensing means may comprise a skin conductance measuring device. The condition sensing means may comprise an electrocardiogram measuring device.

The condition sensing means optionally comprises means for determining neuromuscular dynamics of one or both of the driver’s arms. Advantageously the physical readiness of the driver’s arms is determined.

According to an aspect of the present invention, there is provided a vehicle comprising the system as described above or the control system as described above.

According to an aspect of the present invention, there is provided a method of controlling a transition between an autonomous driving mode and a manual driving mode, the method comprising receiving a condition signal indicative of a condition of a driver of the vehicle, determining a haptic feedback signal indicative of haptic feedback to be applied to a control of the vehicle, wherein the haptic feedback signal is determined in dependence on the condition signal and a desired driver control input for driving the vehicle in the manual mode; and outputting the haptic feedback signal for causing an application of haptic feedback to a control of the vehicle.

The method may comprise receiving a transition signal indicative of a request to transition between the autonomous driving mode and the manual driving mode.

The condition signal may be indicative of an estimate of a cognitive condition of the driver of the vehicle.

The condition signal may be indicative of an estimate of muscle readiness of the driver of the vehicle to assume control of the vehicle in the manual driving mode.

The method optionally comprises determining a desired control authority allocated to the driver of the vehicle in dependence on the condition signal.

The method optionally comprises determining an error indicative of a difference between the desired control authority and a current degree of control authority of the driver of the vehicle, and wherein the haptic feedback signal is determined further in dependence on the error.

The method may comprise receiving an actual control input signal indicative of an actual control input of the driver of the vehicle.

The current degree of control authority may be determined in dependence on the desired control input for the vehicle and the actual control input of the driver of the vehicle.

Determining the haptic feedback signal may comprise determining a first part of the haptic feedback signal in dependence on the condition signal.

Optionally the method comprises receiving an actual control input signal indicative of an actual control input of the driver of the vehicle.

The method may comprise determining a control difference between the desired driver control input for the vehicle and the actual control input of the driver of the vehicle, wherein the haptic feedback signal is determined in dependence thereon.

Determining the haptic feedback signal optionally comprises determining a second part of the haptic feedback signal in dependence on the control difference.

The first part of the haptic feedback signal may be determined as a feedback component of the haptic feedback signal. The second part of the haptic feedback signal may be determined as a feedforward component of the haptic feedback signal.

The condition of a driver of the vehicle is optionally determined at least in part by one or more of image data of the driver, a temperature of at least a portion of the driver’s body, a skin conductance of the driver and electrocardiogram data.

The condition of a driver of the vehicle may be determined at least in part by a signal indicative of neuromuscular dynamics of one or both of the driver’s arms

According to an aspect of the present invention, there is provided computer software which, when executed, is arranged to perform a method as described above. The computer software may be stored on a computer-readable medium. The computer readable medium is optionally non-transitory.

Any controller, controllers or control system described herein may suitably comprise a control unit or computational device having one or more electronic processors. Thus the control system may comprise a single control unit or electronic controller or alternatively different functions of the controller may be embodied in, or hosted in, different control units or controllers. As used herein the term “controller” or “control unit” will be understood to include both a single control unit or controller and a plurality of control units or controllers collectively operating to provide any stated control functionality. To configure a controller, a suitable set of instructions may be provided which, when executed, cause said control unit or computational device to implement the control techniques specified herein. The set of instructions may suitably be embedded in said one or more electronic processors. Alternatively, the set of instructions may be provided as software saved on one or more memory associated with said controller to be executed on said computational device. A first controller may be implemented in software run on one or more processors. One or more other controllers may be implemented in software run on one or more processors, optionally the same one or more processors as the first controller. Other suitable arrangements may also be used.

Within the scope of this application it is expressly intended that the various aspects, embodiments, examples and alternatives set out in the preceding paragraphs, in the claims and/or in the following description and drawings, and in particular the individual features thereof, may be taken independently or in any combination. That is, all embodiments and/or features of any embodiment can be combined in any way and/or combination, unless such features are incompatible. The applicant reserves the right to change any originally filed claim or file any new claim accordingly, including the right to amend any originally filed claim to depend from and/or incorporate any feature of any other claim although not originally claimed in that manner.

BRIEF DESCRIPTION OF THE DRAWINGS

One or more embodiments of the invention will now be described by way of example only, with reference to the accompanying drawings, in which:

Figure 1 shows a schematic illustration of a steering system of a vehicle according to an embodiment of the invention;

Figure 2 shows a haptic take-over unit according to an embodiment of the invention;

Figure 3 shows a haptic take-over unit according to an embodiment of the invention;

Figure 4 illustrates driver control authority against driver workload according to an embodiment of the invention;

Figure 5 illustrates driver control authority against muscle readiness according to an embodiment of the invention;

Figure 6 illustrates a method according to an embodiment of the invention;

Figure 7 illustrates allowed driver authority and actual driver participation according to an embodiment of the invention;

Figure 8 illustrates haptic feedback over time according to an embodiment of the invention;

Figure 9 illustrates driver and automation torque over time according to an embodiment of the invention; and

Figure 10 illustrates a vehicle according to an embodiment of the invention

DETAILED DESCRIPTION

Figure 1 illustrates a steering system 100 of a vehicle. The steering system 100 comprises steering control means 110 for applying input torque to control an angular position of one or more steering means 120. The steering means 120 may be used to control a trajectory of the vehicle, in use. The steering means 120 may be, in the case of a land-going vehicle, one or more steering wheels 120 of the vehicle. In one embodiment, where the vehicle is a fourwheeled vehicle, the steering means 120 comprises at least first and second steerable wheels of the vehicle. In some vehicles four wheels of the vehicle may be steerable particularly at low speeds. In other embodiments, such as where the vehicle is a motorcycle the steering means 120 may be one steerable wheel of the vehicle, or a rudder of the vehicle where the vehicle is a boat or aircraft. The steering control means 110 may be a system for conveying input torque to the steering means 120. The steering control means 110 may be a steering control system 110 comprising mechanical components for converting radial motion to linear motion of one or more linkages to the steering means 120, such as pushrods. The steering control system 110 may comprise a steering rack of the vehicle, although other arrangements are envisaged.

In the illustrated embodiment, the steering control system 110 is arranged to receive inputs from first and second sources 10, 20. In use a driver of the vehicle may be a source 10 of a first input, where the driver’s input is received via a steering control 130 of the vehicle. In some embodiments the steering control 130 is a steering wheel 130 of the vehicle, although other steering controls may be envisaged, such as a joystick. The driver applies a driver steering torque 15 to the steering control system 110 via the steering wheel 130. In the illustrated embodiments an autonomous driving system 20 of the vehicle is a source of a second input to the steering control system 110. The autonomous driving system 20 is arranged to drive the vehicle in an at least partly autonomous driving mode. The at least partly autonomous driving mode may be at least a Level 3 or Level 4 autonomous driving mode, such as defined by SAE International. The autonomous driving system 20 is arranged to receive information about an environment of the vehicle and to determine appropriate inputs to navigate the vehicle within the environment. Some of the inputs correspond to steering inputs, electrical signals indicative of which are provided to one or more steering actuators 140 which may be, for example, electric motors arrange to apply autonomous steering torque to the steering control system 110. In some embodiments, the electrical signals 25 from the autonomous driving system 20 may be considered to be indicative of a desired input 25 for steering the vehicle, as will be explained.

It will be appreciated that other arrangements to that shown in Figure 1 may be envisaged. For example, torque may only be applied to the steering control system 110 by the steering actuator 140 where electrical signals indicative of both the driver’s input and the steering input from the autonomous driving system 20 are provided to control the steering actuator 140.

Figure 2 shows a haptic take-over control (HTOC) unit 200 according to an embodiment of the invention. The HTOC 200 is shown in Figure 2 in relation to the driver 10 and the autonomous driving system 20 of the vehicle. The HTOC unit 200 is a controller 200 for controlling a transition between an autonomous driving mode and a manual driving mode of the vehicle. In some embodiments, the HTOC unit 200 may provide an input to the steering system 100. In some embodiments, the HTOC unit 200 is arranged to provide a haptic feedback signal indicative of haptic feedback to be applied to a control of the vehicle where the haptic feedback is appreciable to the driver. The haptic feedback may be applied to the steering wheel of the vehicle in some embodiments.

As described above, the driver 10 provides an input 15 to the steering system 100 of the vehicle. The driver’s input 15 will be referred to as the driver’s actual input 15 ud to the steering system 100. The driver’s input 15 is generated by a neuromuscular system (NMS) of the driver. The autonomous driving system 20 provides an autonomous input 28 to the steering system 100. The input 28 from the autonomous driving system 20 will be referred to as autonomous input 28 ua to the steering system 100. In embodiments of the invention, during a take-over process between an autonomous driving mode and a manual driving mode where the driver 10 is taking-over from the autonomous driving system 20 or resuming manual driving of the vehicle, the driver’s input 15 takes up a percent of the overall steering input applied to the vehicle where u_A occupies a remaining portion of an optimal steering control input. Once a=100% the driver is manually driving the vehicle i.e. the vehicle is fully operable in the manual mode and the take-over process is completed. It will be appreciated that, even in the manual mode one or more driver assistance systems (ADAS), often referred to as advanced driver assistance systems, may assist the driver in manually driving the vehicle. For example, in some embodiments a lane assist system may provide steering torque to assist the driver in maintaining the vehicle within a driving lane. Therefore, it will be understood that the vehicle being fully operable in the manual mode does not exclude such assistance.

It is desired for the take-over process to be completed in a smooth manner. Therefore, it is not possible to immediately cease automation and handover the control to the human driver 10. Instead, in embodiments of the invention, the driver’s take-over capability is assessed, in real time. It is then determined how much control authority should be allocated to the driver 10. The control authority allocated to the driver may be gradually increased during the take-over process until the driver 10 is fully in control of the steering of the vehicle. In embodiments of the invention the HTOC unit 200 determines a driver control authority Od. The driver control authority Od is determined in embodiments of the invention in dependence on a condition of a driver of the vehicle and a desired driver control input for driving the vehicle in the manual mode.

As illustrated, the HTOC unit 200 may comprise an input for receiving a transition signal 50 indicative of a request to transition between the autonomous driving mode and the manual driving mode. The transition signal 50 may be provided by the autonomous driving system 20, for example in response to determining that a situation exists in which autonomous driving may not be continued. The transition signal 50 may be provided by another source such as, although not exclusively, from the driver 10 themselves indicative of a wish to transition to manual driving of the vehicle.

The HTOC unit 200 is arranged to determine a haptic feedback signal 210 indicative of haptic feedback to be applied to a steering control of the vehicle. The haptic feedback signal 210 may be indicative of a magnitude of the haptic feedback to be applied to the control of the vehicle. In some embodiments, the HTOC unit 200 is arranged to determine a torque control signal 210 indicative of an amount of torque to be applied to a steering control of the vehicle, where the steering control may be the steering wheel 130. The HTOC unit 200 may determine the haptic feedback signal 210 in real-time during the take-over process. In Figure 2 the haptic feedback signal 210 is illustrated as being provided to the driver 10 as the haptic feedback applied to the steering control 130 is perceived by the driver 10 during the take-over process. It will, however, be appreciated that the haptic feedback signal 210 may be provided to a means for controlling the application of toque to the steering control 130 such as a controller for a motor connected to the steering wheel 130 for applying haptic feedback to the steering wheel 130.

Figure 3 schematically illustrates a composition of the HTOC unit 200 according to an embodiment of the invention. The HTOC unit 200 may be implemented by an electronic processor 1110, as shown in Figure 11, which operatively executes computer-readable instructions which may be stored in a memory 1120 accessible to the processor 1110. In some embodiments, the HTOC unit 200 comprises a plurality of functions which each may be implemented in software executing on the processor of the HTOC unit 200.

The HTOC unit 200 comprises a driver control authority allocation (DCAA) means or module 310. The DCAA module 310 is arranged to determine a desired control authority for the driver

10. The desired control authority Od is determined by the DCAA module 310 in dependence on a condition of the driver 10 as will be explained. The DCAA is arranged to output a signal

315 indicative of the desired control authority Qd. The desired control authority Qd may be expressed as a percentage Qd% in some embodiments.

The HTOC unit 200 comprises a driver participation determination (DPD) means or module 320. The DPD module 320 is arranged to determine a current degree of driver participation a in steering the vehicle. The current degree of driver participation may be understood to be an amount of required steering input provided by the driver 10. The degree of driver participation may, in some embodiments, be a value between predetermined minimum or maximum such as 0 and 1 with 0 representing no driver input and 1 representing full driver steering input being provided to cause the vehicle to follow a desired course. The current degree of driver participation a may be determined in dependence on the driver’s actual input 15 ud to the steering system 100 and an expected or desired input 25 for steering the vehicle u_d. Both the driver’s actual input 15 and the desired input 25 may be determined as a sequence of inputs over a predetermined period of time, for example 100ms, 500ms, 1 second, 2 seconds etc. A signal indicative of the desired input 25 for steering the vehicle u_dmay be provided from the autonomous driving system 20. An explanation of determining the desired input 25 for steering the vehicle u_d is provided below.

In some embodiments the driver’s current degree of participation a may be determined as a percentage a% using the equation:

„0/ _ ^ud %

where, as noted above, u_D is the driver’s actual input 15 and u_d is the desired input 25 for steering the vehicle. A signal 325 indicative of the driver’s current degree of participation a% is output by the DPD module 320.

A summation node 330 may therefore determine an error ^between the desired control authority Qd and the actual participation of the driver in steering the vehicle a%. The summation node receives the signal 325 indicative of the driver’s current degree of participation 325 and the desired control authority signal 315. The summation node 330 is arranged to output a participation error signal 335 indicative of e_c.

Returning to the DCAA module 310, as noted above, the driver control authority Qd is determined in dependence on a condition of a driver of the vehicle. The DCAA 310 is arranged to receive at least one signal 345, 355 which is indicative of the condition of the driver 10. The condition of the driver may be one or both of a mental or physical condition of the driver 10. The condition of the driver 10 is understood to be indicative of the driver’s readiness to drive the vehicle in the manual mode. The condition may be one or both of a cognitive state or a muscular state or readiness of the driver 10. The HTOC unit 200 comprises one or both of a cognitive determination module (CDM) 340 and a muscular determination module (MDM) 350. Whilst the illustrated embodiment described herein comprises both the CDM 340 and MDM 350 it will be appreciated that embodiments of the invention may be envisaged with only one of said modules 340, 350.

The CDM 340 is arranged to determine an estimate of the driver’s cognitive workload and to output a signal 345 indicative thereof to the DCAA 310. The signal 345 represents the driver’s mental condition. A driver’s driving performance is expected to be affected by their cognitive workload, which is represented by a value λ. The value λ may adopt a value between a minimum such as 0 and a maximum such as 1 representing the driver’s maximum cognitive ability, although other values may be used. Figure 4 illustrates a model of desired control authority a_d for the driver against cognitive workload according to an embodiment of the invention. The model illustrated in Figure 4 is an inverted-U shape with a maximum desired control authority a_maxat an intermediate value of driver cognitive workload λ. Data indicative of the model may be stored in a memory accessible to the DCAA module 310.

The CDM 340 is arranged to monitor the driver 10 whilst driving the vehicle during the takeover process and to determine the estimate of the cognitive workload of the driver 10. The CDM 340 may receive one or more condition signals 341 relating to the driver which may comprise one or more of image data corresponding to at least a portion of the driver 10, such as the driver’s face, skin conductance, body temperature, heart rate and electrocardiogram signals. The CDM 340 may in some embodiments receive a signal 341 indicative of one or more vehicle states e.g. vehicle position, longitudinal velocity and acceleration, lateral velocity and acceleration. The CDM 340 is arranged to determine the cognitive workload value λ in dependence thereon. The skilled reader will be aware that studies have been conducted in relation to the measurement and estimation of driver cognitive workload using different signal processing and machine learning methods from which the cognitive workload value λ can be determined.

The CDM 340 is arranged to output the cognitive workload signal 345 indicative of the cognitive workload value λ. The DCAA module 310 receives the cognitive workload signal 345 and is arranged to determine a value of cognitive desired control authority Od(A) in dependence thereon. The value of the cognitive desired control authority Od(A) may be determined as:

^ad = (^-^)² where ^ad ^ePT]js the current desired value of driver control authority, φ_Ί and φ₂ are design parameters, a_max is a maximal value for driver control authority (a_max may have a predetermined value, such as 1, to indicate the driver being fully in control of the vehicle), and λ represents the driver’s cognitive workload. In some embodiments, φ₂ is the value of cognitive workload where the desired value of driver control authority Od is maximal i.e. a_max; and φι is indicative of a rate of change of the driver control authority i.e. a gradient of the curve shown in Figure 4.

The MDM 350 is arranged to determine an estimate of the driver’s muscular state or readiness and to output a muscle state signal 355 indicative thereof to the DCAA 310. The muscle state signal 355 represents the driver’s muscular state or readiness to drive the vehicle in the manual mode. A driver’s driving performance is expected to be affected by their muscular state or readiness to drive the vehicle, which impacts their manual driving performance, and is represented by a muscle state value η. The value η may adopt a value between a minimum such as 0 and a maximum such as 1 representing the driver’s maximum muscular readiness, although other values may be used. Figure 5 illustrates a model of desired control authority a_dfor the driver against muscular readiness η according to an embodiment of the invention. The model illustrated in Figure 5 is S-shaped indicating generally increasing desired control authority a_d until a maximum desired control authority a_maxwith increasing muscular readiness η. Data indicative of the model may be stored in a memory accessible to the DCAA module 310.

Muscle state or readiness may be represented by the neuromuscular dynamics of the driver’s arms during steering of the vehicle and may be considered to be nonlinear in nature. During the take-over process, large steering torque may be required to avoid obstacles or to keep the vehicle within the lane, for example. If the driver’s arm muscles are completely relaxed, then the driver’s readiness for the driving task may be considered to be low. Possible reasons that could lead to the above situation may be distraction of the driver 10 by non-driving tasks and a lack of awareness regarding the surrounding environment or effort required. However, since humans have the ability to learn and adapt, muscle readiness may increase, for example exponentially, with time as the driver refocuses on the driving task and completely resumes control. Thus, the control authority a_d may be modelled in embodiments of the invention as a function of the degree of muscle readiness, such as in Figure 5.

In embodiments of the invention, a driver-steering-wheel coupled system may be represented as an inertia model. A dynamic equation for the driver-steering-wheel system may be represented as:

I’d T^is — Jd_sw + 6_SW + K6_SW ^B~^Bdr ^+Bsw where T_D is the driver’s steering input i.e. steering torque in some embodiments (T_D may therefore equal u_Dfor lateral control of the vehicle, although in some embodiments u_D may include other inputs, such as relating to longitudinal control of the vehicle i.e. accelerator and brake inputs), Tdis is a disturbance input, or disturbance torque in some embodiments which is externally applied to the steering system, such as by an uneven road surface. K is a stiffness coefficient of the interacting system, and 6_SW is the steering angle. J is lumped inertia, ty/Js the driver arm’s inertia, J_sw is the inertia of the steering wheel, B_dl and B_sw are the viscous damping terms of the driver’s neuromuscular system and steering wheel bearings, respectively, and B is the lumped viscous coefficient.

Using a vehicle model, such as described below, a transfer function of the driver-vehicle interacting system may be determined. The transfer function according to an embodiment of the invention may be as follows:

&_{sw =} 1

T_D Js² +Bs + K where s indicates a derivative of B and s² indicates a second-order derivative of J.

To characterize the interactions between the driver and steering wheel, key parameters of the transfer function were identified by formulating this system identification as a nonlinear least squares problem, and adopting a Gauss-Newton method as the search algorithm to solve it. Based on the above methodology, steering experiments were carried out using different steering tasks, distinct driver postures and hand griping positions in a driving simulator. Correlations were then used to investigate the relationships between neuromuscular state and the key parameters of the estimated transfer function model in multiple driving activities. The results showed that the parameter of stiffness coefficient K was highly correlated with muscle activity during driving. Thus, the stiffness coefficient K is selected in embodiments of the invention as an indicator of neuromuscular state.

In embodiments of the invention, the real-time degree of muscle readiness can be expressed as:

Factual ^{η =} ~κ--^ref

Where K_actuai and K_ref are reference and actual values of the stiffness coefficient.

In embodiments of the invention K_ref may be determined based on the desired input 25 of the driver. In some embodiments, K_ref may be determined as:

_ u_d - (JO sw + B9_SW)

F_ref - ~ ^usw

The reference value of the muscle stiffness coefficient K_ret can be derived using experimental data. The actual stiffness coefficient Kactuai can be determined in real time based on driver steering torque and steering angle, signals indicative of which are provided to the MDM 350 as indicated by input which is arranged to received condition signal 351 indicative of a neuromuscular condition of the driver 10. In some embodiments, /Cac^a/may be determined as:

_ u_D - (J9

SW + ^B9sw) ^actual ~ n “sw

The MDM 350 is arranged to monitor the driver 10 whilst driving the vehicle during the takeover process and to determine the estimate of the muscle readiness of the driver 10. The MDM 350 is arranged to output the muscle state signal 355 indicative of the muscle state value η. The DCAA module 310 receives the muscle state signal 355 and is arranged to determine a value of muscle state desired control authority Od(q) in dependence thereon.

In some embodiments, the DCAA module 310 determines the value of the muscle state desired control authority Od(q) according to:

α_α(η) = 1 -μ^16 where μι and μ2 are parameters which may be selected appropriately. In some embodiments μι and μ2 may have the same value, although this isn’t necessary and may have different values. For example, μι and μ2 may be between 5 and 15 and, in one embodiment, μι = //2=10.

The DCAA module 310 may use only one of the cognitive workload value λ to determine the cognitive desired control authority Od(A) and the muscle state value η to determine the muscle state desired control authority Od(q), since in some embodiments only one of the signals 345, 355 may be received by the DCAA module 310. In such embodiments an overall desired control authority Od is equal to the respective one of Od(A) or Qd(q).

However, in some embodiments, where both signals 345, 355 are received, the DCAA module 310 determines both the cognitive desired control authority a_d(A) and the muscle state desired control authority a_d(q) which are combined to determine an overall desired control authority □d. The overall desired control authority Od may be determined by the DCAA module 310 in dependence on the cognitive desired control authority a_d(A) and the muscle state desired control authority a_d(q) values which may be appropriately weighted with respecting weighting values. In one embodiment, the overall desired control authority a_d may be determined according to:

Xd=<*d (η)

Although it will be appreciated that other functions may be used to combine the respective control authority values. The DCAA module 310 is arranged to output the desired control authority signal 315 indicative of the determined value Od. The desired control authority signal 315 is provided to summation node 330 as described above to determine the participation error signal 335 which may in some embodiments be defined as:

e_c = a_d% — a°/o

The HTOC module 200 comprises a haptic feedback (HF) module 360. The HF module 360 is arranged to receive the participation error signal 335 from the summation node 330. The HF module 360 is arranged to determine one or more attributes of haptic feedback to be applied to a control of the vehicle in dependence thereon. The HF module 360 is arranged to output a haptic feedback signal 365 indicative of the one or more attributes of haptic feedback. In some embodiments, the haptic feedback signal 365 is indicative of a magnitude of the haptic feedback. In some embodiments, the haptic feedback signal may be indicative of a direction of the haptic feedback to be applied to the control of the vehicle. In embodiments where the control is a rotary control of the vehicle, such as the steering wheel, the haptic feedback signal 365 may be indicative of a torque value representing an amount of torque to be applied to the control of the vehicle i.e. the steering wheel. The torque may be applied, for example, by a motor associated with the control such as a motor associated with the steering wheel of the vehicle. The torque may be applied in a direction consistent with a current desired control input of the vehicle. For example, if the steering is intended to be turned in a clockwise direction, the torque may be applied in the clockwise direction.

In some embodiments, as the haptic feedback is determined in dependence on the participation error signal 335, the haptic feedback is determined to assist the driver in achieving an optimal control input for the vehicle. The haptic feedback signal 365 may be indicative of the deviation between a % and the desired control authority 6(/%. which advantageously assists the driver in completing the take-over process.

The HF module 360 may determine the haptic feedback based upon a feedforward and a feedback portion. The HF module 360 in some embodiments is arranged to determine the haptic feedback Thpt, of which the haptic feedback signal 365 is indicative as:

Thpt — Tff + Tf_b

Where T_ff is the feedforward portion and T_fb is the feedback portion of the haptic feedback. The T is used to denote torque, although it will be noted that in other embodiments the haptic feedback signal 365 may be indicative of linear force rather than torque.

The feedforward portion T_ff may be determined in dependence on the driver’s actual input ud and the desired input for steering the vehicle Ud· In some embodiments, the feedforward portion corresponds to an assistance input ua which is indicative of a difference between the desired input for steering the vehicle and the driver’s actual input, as follows:

Tff ^Ud ^UD ^UA

In some embodiments, the feedback portion T_fb is based on the error ^between the desired control authority Od and the actual participation of the driver in steering the vehicle. The feedback portion may be determined as follows:

K_pe_c Kb J e_cdt where K_p and Ki are gain values.

Figure 6 illustrates a method 600 according to an embodiment of the invention. The method 600 is a method of controlling a transition between an autonomous driving mode and a manual driving mode of a vehicle. The method 600 may be performed by a control system according to an embodiment of the invention, such as the HTOC unit 200 illustrated in Figure 2. The method 600 may be implemented by one or more electronic processors.

The method 600 may be initiated in dependence on receiving 610 a transition signal indicative of a request to transition between the autonomous driving mode and the manual driving mode. The transition signal 50 may be output by an autonomous driving controller of the vehicle such as in response to the autonomous driving controller determining that one or more conditions exist which lead to autonomous driving being unable to continue. The transition signal 50 may alternatively originate from the driver 10 providing an indication that they wish to assume manual driving of the vehicle.

The method 600 comprises a step 620 of receiving a condition signal indicative of a condition of a driver of the vehicle. The condition signal may be output by one or more condition sensing devices, as described below.

The method comprises a step 630 of determining a haptic feedback signal indicative of haptic feedback to be applied to a control of the vehicle. The haptic feedback signal is determined in dependence on the condition signal and a desired driver control input for driving the vehicle in the manual mode. The haptic feedback signal may be determined as described above.

The method comprises a step 640 of outputting the haptic feedback signal for causing an application of haptic feedback to a control of the vehicle. The haptic feedback signal may be provided to one or more actuators for applying the haptic feedback to the control of the vehicle. Where the control is a rotary control, the haptic feedback may be torque applied to the rotary control such as the steering wheel of the vehicle. The one or more actuators may be, for example, a motor associated with the steering system of the vehicle.

As noted above, in order to determine the desired input 25, which may be considered as an optimal input sequence, the driver-automation collaboration system, along with vehicle dynamics, may be modeled. Thus a model of the vehicle may be used to determine the desired input Ud 25 described above. A 2-degree-of-freedom (DOF) linearized bicycle model may be adopted to approximate the vehicle’s dynamic behavior. It will be appreciated that other models may be used and that embodiments of the present invention are not limited in this respect. Assuming that the longitudinal velocity is constant at V_x, then the power steering system of the vehicle can be described by the following state space :

i(r) ~ Jx(?)+(?) z(i) ™ Cx(<) where: the ⁼ L^V’^W

V (?) ^y is the lateral velocity, ®(f) is the yaw rate, is the lateral displacement, is the yaw angle, and ⁵' is the angular position of the steering wheel. The steering torque ^{r<l> T}D(0+T_A(t) j_{S S}y_St_em input, ^T° is the torque applied by the human driver, and ^T° is the torque contributed by automation system.^(/) is the output of the system. ,and C are constant continuous-time matrices, which can be represented by:

	’ -(C_/+C_r) ml;	-{aC_f-bC ) _:—L---—-V mV_x	0	0	S	0
	-(aC_f-bC_r)	-(a²Cf+b²Cr)	0	0		0
	/χ	iy_x	X
A =	1	0	0	u	0	0
	0	1	0	0	0	0
	0	0	0	0	0	1
	0	0	0	0	-K	-B

B= 0 0 0 0 0

and

C = [o 0 1 0 0 o] where m is the total mass of the vehicle, Cr is the front cornering stiffness, C_r is the rear cornering stiffness, m is the vehicle mass, a is the distance from the center of mass to the front axle, b is the distance from the center of mass to the rear axle, l_z is the polar moment of inertia, and /_s is the transmission ratio of the steering system. \Z_X is the longitudinal velocity with a constant value.

By further discretizing the above state space with a sampling time of through the zeroorder hold method, then the discrete-time representation of the system model can be shown by:

x(k +1) = A_dx(k) + B_du(k) = C_dxQ0

A B = Γ e^ATdr ·B p ₌ p where ^d , ^{d Jo} , and are the discrete matrices.

In order to demonstrate an effective of embodiments of the invention simulations were undertaken. In these simulations, a longitudinal speed of the vehicle was set to a constant 30 km/h. Some key parameters adopted in the simulations are listed in the table below, and the main results are described below.

Vehicle mass	1360	kg
Wheel base	2.33	m
Frontal area of the vehicle	2.142	m²
Coefficient of air resistance	0.32	-
Front wheel cornering stiffness	0.275	N/rad
Rear wheel cornering stiffness	3.79	-
Polar moment of inertia	1750	kgm²
Steering ratio	15	-
Longitudinal velocity	30	km/h

After receiving the take-over request, the driver shifts their attention back to the driving task, and the cognitive workload and muscle readiness are assumed to rapidly recover. Thus, the allowed take-over control authority for the driver (a_d) increases from 0 to 100% within 3 seconds, as illustrated in Figure 7 which illustrates this against the actual control authority a% against time.

Figure 8 illustrates haptic feedback over time. Illustrated in Figure 8 is Torque (Nm) against time. It will be appreciated that embodiments of the invention are not restricted to torque as the haptic feedback.

During the take-over transition process, in some embodiments the haptic feedback torque is applied to the steering wheel and dynamically adjusted, as shown in Figure 8, so as to minimize the deviation between the actual proportion of driver input and the desired control authority. Advantageously this results in a smooth transition from automatic control to manual driving.

Figure 9 illustrates values of ua and ud during the take-over process. Figure 9 shows that during the first 1,5s the driver is not well prepared for the take-over process, due to the workload and the neuromuscular state, and so the automation system dominates the overall steering torque input. After this time period, the driver gradually applies more torque, under the assistance of the generated haptic feedback, and the contribution of the system decreases accordingly. Figure 9 also shows that the handover transition process is completed at around 3s, although embodiments of the invention are not limited in this respect. During the whole take-over control procedure, the actual value of the total torque contributed by the driver and automation fits the required control input sequence very well (see Figure 9), indicating the feasibility and effectiveness of the haptic take-over assistance according to embodiments of the invention.

Figure 10 illustrates a vehicle 1000 according to an embodiment of the invention. The vehicle may be a land-going vehicle comprising a plurality of wheels. The vehicle may comprise a controller according to an embodiment of the invention such as the HTOC unit 200 illustrated above, ora system comprising the controller200. The system may comprise condition sensing means, such as one or more condition sensing devices, for determining a condition of a driver of the vehicle and providing the condition signal to the HTOC unit 200 in dependence thereon. The condition sensing device may comprise one or more of an imaging device for providing image data of the driver, temperature monitoring device for determining a temperature of at least a portion of the driver’s body, a skin conductance measuring device and an electrocardiogram measuring device. The condition sensing device may be arranged to for determine neuromuscular dynamics of one or both of the driver’s arms.

Figure 11 illustrates a controller 1100 for controlling a transition between an autonomous driving mode and a manual driving mode according to an embodiment of the invention. Although one controller is shown, it will be appreciated that embodiments of the invention may be embodied comprising a plurality of controllers. The one or more controllers 1100 collectively comprise at least one electronic processor 1110 and an electrical input 1130 for receiving a condition signal. The one or more controllers comprise at least one memory device 1120 coupled to the at least one electronic processor 1110 and having instructions stored therein. The electronic processor 1110 is configured to access the memory device 1120 and execute the instructions stored therein, so as to determine the haptic feedback signal. The one or more controllers comprise an electrical output 1140 for outputting the haptic feedback signal.

It will be appreciated that embodiments of the present invention can be realised in the form of hardware, software or a combination of hardware and software. Any such software may be stored in the form of volatile or non-volatile storage such as, for example, a storage device like a ROM, whether erasable or rewritable or not, or in the form of memory such as, for example, RAM, memory chips, device or integrated circuits or on an optically or magnetically readable medium such as, for example, a CD, DVD, magnetic disk or magnetic tape. It will be appreciated that the storage devices and storage media are embodiments of machinereadable storage that are suitable for storing a program or programs that, when executed, implement embodiments of the present invention. Accordingly, embodiments provide a program comprising code for implementing a system or method as claimed in any preceding claim and a machine readable storage storing such a program. Still further, embodiments of the present invention may be conveyed electronically via any medium such as a communication signal carried over a wired or wireless connection and embodiments suitably encompass the same.

All of the features disclosed in this specification (including any accompanying claims, abstract and drawings), and/or all of the steps of any method or process so disclosed, may be combined in any combination, except combinations where at least some of such features and/or steps are mutually exclusive.

Each feature disclosed in this specification (including any accompanying claims, abstract and drawings), may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise. Thus, unless expressly stated otherwise, each feature disclosed is one example only of a generic series of equivalent or similar features.

The invention is not restricted to the details of any foregoing embodiments. The invention extends to any novel one, or any novel combination, of the features disclosed in this specification (including any accompanying claims, abstract and drawings), or to any novel one, or any novel combination, of the steps of any method or process so disclosed. The claims should not be construed to cover merely the foregoing embodiments, but also any embodiments which fall within the scope of the claims.

Claims

1. A control system for controlling a transition between an autonomous driving mode and a manual driving mode of a vehicle, the control system comprising one or more controllers, configured to:

receive a condition signal indicative of a condition of a driver of the vehicle;

determine a haptic feedback signal indicative of haptic feedback to be applied to a control of the vehicle, wherein the haptic feedback signal is determined in dependence on the condition signal and a desired driver control input for driving the vehicle in the manual mode; and output the haptic feedback signal to cause an application of haptic feedback to a control of the vehicle.

2. The control system of claim 1, wherein the one or more controllers collectively comprise:

at least one electronic processor having an electrical input for receiving the condition signal; and at least one memory device coupled to the at least one electronic processor and having instructions stored therein;

wherein the at least one electronic processor is configured to access the at least one memory device and execute the instructions stored therein so as to determine the haptic feedback signal.

3. The control system of claim 1 or claim 2, wherein the condition signal is indicative of an estimate of a cognitive condition of the driver of the vehicle.

4. The control system of any preceding claim, wherein the condition signal is indicative of an estimate of muscle readiness of the driver of the vehicle to assume control of the vehicle in the manual driving mode.

5. The control system of any preceding claim, arranged to determine the haptic feedback signal having a magnitude determined in dependence on the condition signal and a direction determined in dependence on desired driver control input.

6. The control system of any preceding claim, arranged to determine a desired control authority allocated to the driver of the vehicle in dependence on the condition signal.

7. The control system of claim 6, arranged to determine an error indicative of a difference between the desired control authority and a current degree of control authority of the driver of the vehicle, and to determine the haptic feedback signal in dependence thereon.

8. The control system of claim 7, configured to:

receive an actual control input signal indicative of an actual control input of the driver of the vehicle; and determine the current degree of control authority in dependence on the desired control input for the vehicle and the actual control input of the driver of the vehicle.

9. The control system of any preceding claim, configured to determine a first part of the haptic feedback signal in dependence on the condition signal.

10. The control system of any preceding claim, configured to:

receive an actual control input signal indicative of an actual control input of the driver of the vehicle;

determine a control difference between the desired driver control input for the vehicle and the actual control input of the driver of the vehicle; and determine the haptic feedback signal in dependence on the control difference.

11. The control system of claim 10, configured to determine a second part of the haptic feedback signal in dependence on the control difference.

12. The control system of any of claims 9 to 11, configured to determine the first part of the haptic feedback signal as a feedback component of the haptic feedback signal and the second part of the haptic feedback signal as a feedforward component of the haptic feedback signal.

13. A system, comprising:

the control system of any preceding claim; and condition sensing means for determining a condition of a driver of the vehicle and providing the condition signal to the control system in dependence thereon.

14. The system of claim 13, wherein the condition sensing means comprises one or more of an imaging device for providing image data of the driver, temperature monitoring device for determining a temperature of at least a portion of the driver’s body, a skin conductance measuring device and an electrocardiogram measuring device.

15. The system of claim 13 or 14, wherein the condition sensing means comprises means for determining neuromuscular dynamics of one or both of the driver’s arms.

16. A vehicle comprising the control system of any of claims 1 to 12 or the system of any of claims 13 to 15.

17. A method of controlling a transition between an autonomous driving mode and a manual driving mode, the method comprising:

receiving a condition signal indicative of a condition of a driver of the vehicle;

determining a haptic feedback signal indicative of haptic feedback to be applied to a control of the vehicle, wherein the haptic feedback signal is determined in dependence on the condition signal and a desired driver control input for driving the vehicle in the manual mode; and outputting the haptic feedback signal for causing an application of haptic feedback to a control of the vehicle.

18. The method of claim 17, wherein the condition signal is indicative of: an estimate of a cognitive condition of the driver of the vehicle; and/or an estimate of muscle readiness of the driver of the vehicle to assume control of the vehicle in the manual driving mode.

19. The method of claim 17 or 18, wherein the method comprises determining a desired control authority allocated to the driver of the vehicle in dependence on the condition signal.

20. The method of claim 19, wherein the method comprises determining an error indicative of a difference between the desired control authority and a current degree of control authority of the driver of the vehicle, and wherein the haptic feedback signal is determined further in dependence on the error.

21. The method of claim 20 wherein the method comprises receiving an actual control input signal indicative of an actual control input of the driver of the vehicle; and the current degree of control authority is determined in dependence on the desired control input for the vehicle and the actual control input of the driver of the vehicle.

22. The method of any of claims 17 to 21 wherein determining the haptic feedback signal comprises determining a first part of the haptic feedback signal in dependence on the condition signal.

23. The method of any of claims 17 to 22, wherein the method comprises:

receiving an actual control input signal indicative of an actual control input of the driver of the vehicle; and determining a control difference between the desired driver control input for the vehicle and the actual control input of the driver of the vehicle, wherein the haptic feedback signal is determined in dependence thereon.

24. The method of any of claims 17 to 23 wherein the condition of a driver of the vehicle is determined at least in part by a signal indicative of neuromuscular dynamics of one or both of the driver’s arms

25. Computer software which, when executed, is arranged to perform a method according to any of claims 17 to 24; optionally the computer software being stored on a computerreadable medium.