KR102592102B1 - Apparatus and method for determining readiness for autonomous driving control transfer - Google Patents

Abstract

The present invention relates to a device and method for determining readiness for switching control of autonomous driving.
The readiness determination device for switching autonomous driving control according to the present invention includes a recognition unit that acquires status information of the driver inside the vehicle and obtains driving-related information of the vehicle, and a recognition unit that acquires driving-related information of the vehicle, and uses the driver's status information and driving-related information to determine driver status. Calculates the safe driving capability index, generates a warning according to the judgment unit and safe driving capability index that determines whether driving control can be switched according to the safe driving capability index, and monitors changes in the driver's condition following the occurrence of the warning. It includes a control unit that transmits a control command according to a preset risk cancellation strategy.

Description

Device and method for determining readiness for autonomous driving control transfer {APPARATUS AND METHOD FOR DETERMINING READINESS FOR AUTONOMOUS DRIVING CONTROL TRANSFER}

The present invention relates to an apparatus and method for determining readiness for switching control of autonomous driving.

In the case of manual driving mode where the driver drives, poor driving posture is an important factor in causing accidents.

According to the prior art, a technology that monitors drowsy driving situations and provides a warning to the driver, and a technology that provides a warning to the driver in situations where the steering wheel is not gripped as a type of ADAS function has been proposed, but the driver's posture information There is a limitation in that it cannot monitor and determine whether the driving posture is normal and provide a hazard warning to the driver.

Additionally, with the advancement of autonomous driving technology, situations where switching between manual driving mode and autonomous driving mode often occur, and in cases where it is necessary to change from autonomous driving mode to manual driving mode, driving to a driver who is not in the correct posture is necessary. When control is transferred, there is a problem that the risk of accidents increases.

The present invention was proposed to solve the above-mentioned problems. It checks the driver's driving posture and compares it with the driving posture model built for each driver while driving, such as other actions other than the normal driving posture (e.g., driving with one hand, drinking a drink). , upper body leaning, seat tilting, etc.), determines the level of risk and provides warnings, and determines whether the driver can take over control and drive safely when switching from autonomous driving mode to manual driving mode (e.g. Safe driving by determining whether or not to transfer driving control by judging the driver's posture (including whether or not the steering wheel is held, the position of the steering wheel, the pressure of the steering wheel, the position and pressure of the feet to apply the brakes, the driver's head direction and gaze), etc. The purpose is to provide devices and methods that promote .

The readiness determination device for switching autonomous driving control according to the present invention includes a recognition unit that acquires status information of the driver inside the vehicle and obtains driving-related information of the vehicle, and a recognition unit that acquires driving-related information of the vehicle, and uses the driver's status information and driving-related information to determine driver status. Calculates the safe driving capability index, generates a warning according to the judgment unit and safe driving capability index that determines whether driving control can be switched according to the safe driving capability index, and monitors changes in the driver's condition following the occurrence of the warning. It includes a control unit that transmits a control command according to a preset risk cancellation strategy.

The control unit generates a warning with a posture correction guide for safe driving in driver control mode, and generates a warning with the driver's posture correction guide prior to switching control in autonomous driving mode.

The recognition unit acquires the driver's face image using the image acquisition unit, and the determination unit uses the face image to determine driving concentration by considering the driver's gaze information.

The recognition unit acquires the status information related to the driver's posture using at least one of sensing information from a plurality of pressure sensors placed on the seat and setting information for the electric seat, and the determination unit uses the status information related to the driver's posture. This determines whether the driving posture is normal.

The recognition unit obtains status information related to the driver's grip on the steering wheel using at least one of sensing information from a contact sensor array disposed on the steering wheel and image information acquired using a camera, and the determination unit obtains status information related to the driver's grip on the steering wheel. Status information is used to determine whether the driving posture is normal.

The recognition unit acquires the status information related to the position and strength of the driver's feet using at least one of sensing information from a capacitive or pressure-sensitive pressure sensor array disposed on the driver's seat mat and image information obtained using a camera, The determination unit determines whether the driving posture is normal using status information related to the position and force of the driver's feet.

The recognition unit receives driving information including at least one of gear, vehicle speed, RPM, and parking brake information from the vehicle sensor, receives control status information, and receives control transfer request status information, and the determination unit receives driving information and control status. Using the information and control transfer request status information, a safe driving possibility index is calculated considering the driver's status information at the preset time, and if the safe driving possibility index is higher than the preset value, it is judged that switching to the driver control mode is possible. .

The judgment unit calculates the driver's safe driving capability index using a previously learned driving posture model for each situation, and if the safe driving capability index is less than a preset value, it requests warning control from the control unit.

The judgment unit requests warning control when the duration of time that the safe driving possibility index is less than the preset value is longer than the preset time.

The judgment unit compares the previously learned normal state driving posture data with the current driver's state information, and determines whether at least one of the minimum, maximum, and average values of the safe driving possibility index is included within the data category of the normal state capable of safe driving. It is determined whether the driving control right can be switched.

Following the warning provided by the control unit, the determination unit updates the safe driving capability index using the driver's status change information obtained by the recognition unit, and presets the updated safe driving capability index to be less than a preset value. If it persists for more than an hour, a control request is sent according to the risk minimization strategy.

When receiving a control request according to a risk minimization strategy, the control unit transmits a control command for parking and stopping the vehicle in consideration of driving-related information including driving path information and the map construction state.

The control unit provides a warning along with a posture correction guide for safe driving using at least one of the in-vehicle speaker, display, and steering.

The method for determining the readiness for switching to autonomous driving control according to the present invention includes the steps of (a) acquiring the driver's status information and driving-related information of the vehicle, and (b) using the driver's status information and driving-related information to determine the driver's status. Calculating a safe driving capability index and determining whether driving control can be switched, and (c) using the safe driving capability index, in the case of a driver control mode or in the case of a control switching request in autonomous driving mode, along with a guide for correcting the driver's posture. It includes the step of generating a warning and transmitting a control command according to a risk minimization strategy by considering the driver's status change information obtained after the warning is issued.

Step (a) is at least one of sensing values obtained using a plurality of pressure sensors placed on the seat, electric seat settings, sensing values of a contact sensor array placed on the steering wheel, and image data acquired using a camera. The above data is combined to obtain the status information related to the driver's posture, and in step (b), a safe driving possibility index is calculated using the status information related to the driver's posture.

In step (b), a safe driving possibility index is calculated using a pre-learned driving posture model for each driving situation, and if the safe driving possibility index is less than a preset value, warning control is requested.

Step (b) compares the pre-learned normal state driving posture data with the current driver's state information, and selects at least one of the minimum, maximum, and average values of the safe driving index within the data category of the normal state where safe driving is possible. Determine whether is included.

Step (c) updates the safe driving ability index calculated in step (b) by considering information on changes in the driver's status, and if the updated safe driving ability index remains below the preset value for more than a preset time, there is a risk. Transmit control commands according to the minimization strategy.

Step (c) performs parking and stopping control for the vehicle by considering driving-related information including driving path information and map construction status.

According to the present invention, safe driving is supported by determining whether each driver has a dangerous posture or behavior, and when switching driving control, rather than simply considering whether or not to look ahead, the driver's posture information (whether or not the driver is gripping the steering wheel) is checked to determine whether or not control is transferred. By determining , it has the effect of supporting safe driving.

The effects of the present invention are not limited to those mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below.

Figure 1 illustrates a device for determining readiness for switching control of autonomous driving according to an embodiment of the present invention.
Figure 2 shows driving posture classification according to an embodiment of the present invention.
Figure 3 shows a driver posture determination and warning process according to an embodiment of the present invention.
Figure 4 shows a control transfer process according to an embodiment of the present invention.

The above-mentioned object and other objects, advantages and features of the present invention, and methods for achieving them will become clear by referring to the embodiments described in detail below along with the accompanying drawings.

However, the present invention is not limited to the embodiments disclosed below and can be implemented in various different forms. The following embodiments are merely intended to convey to those skilled in the art the purpose of the invention, It is only provided to easily inform the configuration and effect, and the scope of rights of the present invention is defined by the description of the claims.

Meanwhile, the terms used in this specification are for describing embodiments and are not intended to limit the present invention. As used herein, singular forms also include plural forms, unless specifically stated otherwise in the context. As used in the specification, “comprises” and/or “comprising” means that the mentioned element, step, operation and/or element precludes the presence of one or more other elements, steps, operations and/or elements. Or it is not excluded that it is added.

Figure 1 illustrates a device for determining readiness for switching control of autonomous driving according to an embodiment of the present invention.

The device for determining readiness for switching control of autonomous driving according to an embodiment of the present invention includes a recognition unit 110 that acquires status information of the driver inside the vehicle and information related to driving of the vehicle, the driver's status information and driving The driver's safe driving capability index is calculated using the relevant information, and a judgment unit 120 determines whether driving control can be switched according to the safe driving capability index, and generates a warning according to the safe driving capability index, and responds to the occurrence of the warning. It includes a control unit 130 that monitors changes in the status of subsequent drivers and transmits control commands according to a preset risk cancellation strategy.

The control unit 130 generates a warning along with a posture correction guide for safe driving in the driver control mode, and generates a warning along with the driver's posture correction guide prior to switching control in the autonomous driving mode.

The recognition unit 110 acquires the driver's face image using the image acquisition unit, and the determination unit uses the face image to determine driving concentration by considering the driver's gaze information.

The recognition unit 110 acquires the status information related to the driver's posture by combining at least one data of the sensing information of the pressure sensor disposed on the seat and the setting information for the electric seat, and the determination unit determines the status related to the driver's posture. The information is used to determine whether the driving posture is normal.

Figure 2 shows driving posture classification according to an embodiment of the present invention.

According to an embodiment of the present invention, it is confirmed whether the driving posture is normal using the handle gripping position, handle gripping pressure, positions of both feet, seat back angle, seat cushion angle, and seat position information.

As shown in Figure 2, driving postures are classified into forward leaning, one-handed driving, poor driving posture, straight posture, drinking, checking rear seat passengers, operating devices, poor two-handed driving, seat folding, etc.

The recognition unit 110 acquires status information related to the driver's grip on the steering wheel using at least one of sensing information from a contact sensor array disposed on the steering wheel and image information obtained using a camera, and the determination unit determines the steering wheel. Status information related to wheel grip is used to determine whether the driving posture is normal.

The recognition unit 110 acquires the status information related to the position and strength of the driver's feet using at least one of sensing information from a capacitive or pressure-sensitive pressure sensor array disposed on the driver's seat mat and image information obtained using a camera. And, the determination unit 120 determines whether the driving posture is normal using status information related to the position and force of the driver's feet.

The recognition unit 110 receives driving information including at least one of gear, vehicle speed, RPM, and parking brake information from the vehicle sensor, receives control status information, and receives control transfer request status information, and the determination unit receives driving information. Using the information, control status information, and control transfer request status information, a safe driving possibility index is calculated considering the driver's status information at the preset time, and if the safe driving possibility index is higher than the preset value, the transition to driver control mode is performed. It is judged to be possible.

The recognition unit 110 recognizes the driver's posture information, personal authentication information, and vehicle status information.

The camera 111 authenticates each individual's face and obtains the driver's face and gaze position information, upper body posture information, ROI area, and average pixel value.

The first pressure sensor 112 is a pressure sensor for a seat, and is configured in a form that allows posture discrimination, such as a driver's seat mat or a seat mat.

The second pressure sensor 113 is a pressure sensor for the handle, detects the handle gripping position and pressure, and may be configured as an array of capacitive sensors.

The vehicle sensor 114 checks driving and stopping status information through gear, vehicle speed, RPM, and parking brake information, checks the control status (automatic/manual driving), and checks the control transfer request status.

The determination unit 120 calculates the driver's safe driving capability index using a pre-learned driving posture model for each situation, and when the safe driving capability index is less than a preset value, it requests warning control from the control unit 130.

The determination unit 120 requests warning control when the duration of time that the safe driving possibility index is less than a preset value is longer than the preset time.

The determination unit 120 compares the pre-learned normal state driving posture data with the current driver's state information, and at least one of the minimum, maximum, and average values of the safe driving possibility index within the data category of the normal state in which safe driving is possible. By determining whether one is included, it is determined whether driving control rights can be switched.

Following the warning provided by the control unit 130, the determination unit 120 updates the safe driving capability index using the driver's status change information obtained by the recognition unit 110, and updates the safe driving capability index. If the value remains below the preset value for more than the preset time, a control request according to the risk minimization strategy is sent.

The driving posture learning and determination unit 121 communicates with the database 140, stores driving posture information, and compares the currently acquired driving posture information with previously collected normal posture information to determine whether to request a warning.

The driving posture learning and determination unit 121 uses data acquired from the camera 111, the first pressure sensor 112, the second pressure sensor 113, and the vehicle sensor 114 to create a driving posture model for each situation. It learns, determines the driving posture through this, and classifies normal and abnormal driving postures.

The driving posture learning and determination unit 121 monitors whether the abnormal driving posture is maintained for more than a preset time and transmits a warning request to the warning generation unit 131.

The driving posture learning and determination unit 121 constructs data for each driver authenticated through the camera 111 and stores it in the individual driving posture DB 140.

In the future, this will be optimized into a highly accurate driving posture judgment model through annotation and inspection for learning through DB operators or business operators.

When switching from autonomous driving mode to manual driving mode, the control switching model 122 statistically compares normal driving posture data and current driving posture data (at this time, comparison is performed through min, max, and avg values), and manually Determine whether the driving position is appropriate for transferring control to driving mode.

If the driver's current posture is not appropriate for transferring control to the manual driving mode, the control transfer model 122 requests the warning generator 131 to issue a warning.

At this time, the control transfer model 122 requests a warning to be issued to request specific modifications as to which body part should be changed and in what manner.

If an inappropriate driving preparation posture is maintained for more than a preset time, the control transfer model 122 determines that the driver is unable to drive manually and requests a control command from the MRM (Minimum Risk Maneuver) control unit 132. .

When receiving a control request according to a risk minimization strategy, the control unit 130 transmits a control command for parking the vehicle in consideration of driving-related information including driving path information and the map construction state.

The control unit 130 provides a warning along with a posture correction guide for safe driving using at least one of the in-vehicle speaker, display, and steering.

The warning generator 131 provides a warning to the driver to induce a normal driving posture through speakers and displays in the vehicle, and provides a warning when the driver is not properly prepared for the manual driving mode when switching control.

The MRM control unit 132 is a component of the autonomous driving system and performs functions such as stopping on the shoulder and parking in consideration of the current driving path and map construction status of the autonomous driving system at the request of the control transfer model 122.

A method of determining readiness for switching control of autonomous driving according to an embodiment of the present invention includes (a) obtaining driver status information and vehicle driving-related information, and (b) using the driver's status information and driving-related information. calculating the driver's safe driving capability index and determining whether driving control can be switched; and (c) using the safe driving capability index to correct the driver's posture in the case of driver control mode or in the case of a request for switching control in autonomous driving mode. It includes the steps of generating a warning with a guide and transmitting a control command according to a risk minimization strategy by considering information on changes in the driver's state obtained after generating the warning.

Step (a) involves at least one of the sensing values obtained using a pressure sensor placed on the seat, the electric seat setting value, the sensing value of the contact sensor array placed on the steering wheel, and the image data acquired using the camera. The state information related to the driver's posture is obtained using the above state information, and in step (b), a safe driving possibility index is calculated using the state information related to the driver's posture.

In step (b), a safe driving possibility index is calculated using a pre-learned driving posture model for each driving situation, and if the safe driving possibility index is less than a preset value, warning control is requested.

Step (b) compares the pre-learned normal state driving posture data with the current driver's state information, and selects at least one of the minimum, maximum, and average values of the safe driving index within the data category of the normal state where safe driving is possible. Determine whether is included.

Step (c) updates the safe driving ability index calculated in step (b) by considering information on changes in the driver's status, and if the updated safe driving ability index remains below the preset value for more than a preset time, there is a risk. Transmit control commands according to the minimization strategy.

Step (c) performs parking and stopping control for the vehicle by considering driving-related information including driving path information and map construction status.

Hereinafter, with reference to FIGS. 3 and 4, the driver posture determination and warning process and the control transfer process according to an embodiment of the present invention will be described.

Figure 3 shows a driver posture determination and warning process according to an embodiment of the present invention.

Step S310 collects driving information including image information and posture information, and step S320 collects and stores seat information for each individual posture.

Step S330 collects seat information for each individual posture and determines whether the driving posture is normal.

If it is determined that the driving posture is normal in step S330, driving information collection continues, and if it is determined that the driving posture is not normal, it is checked whether the abnormal driving posture is maintained for more than a preset time (S340).

If it is confirmed that the abnormal driving posture is maintained for more than a preset time in step S340, a danger warning is provided (S350).

Figure 4 shows a control transfer process according to an embodiment of the present invention.

Step S410 collects driver information and vehicle information in normal manual driving mode.

At this time, in a manual driving mode in which the driver normally has driving control, individual images, steering information, braking information, and seat-related information are classified based on vehicle data, and driving posture data for each situation is collected and stored.

Step S420 checks whether there is a request for a control right change, and at this time, it checks whether the control right change is a situation where the autonomous driving mode is switched to the manual driving mode.

If it is confirmed in step S420 that a control change is necessary, step S430 collects current video, steering information, braking information, and seat-related information.

In step S440, based on the current driving information (video, steering information, braking information, seat-related information), it is determined whether the driver is ready to respond to the manual driving mode through the learned driving posture judgment model.

If it is determined that switching control is appropriate in step S440, step S450 switches control to manual operation mode.

At this time, if the consistency with the normal driving posture model is above a certain value (e.g., 80% or more) using the current image, steering information, braking information, and seat-related information, control transfer is performed.

If it is determined in step S440 that transfer of control is not appropriate, step S460 provides a warning to the driver.

Despite providing a driver warning in step S460, if a situation other than the normal driving posture persists for more than a preset time, MRM control is performed.

In other words, if the consistency of the current driving posture data compared to the driving posture data established during manual driving is less than a certain value, control transfer is not performed and MRM is performed considering the autonomous driving system status.

Meanwhile, the method for determining readiness for switching control of autonomous driving according to an embodiment of the present invention may be implemented in a computer system or recorded on a recording medium. A computer system may include at least one processor, memory, user input device, data communication bus, user output device, and storage. Each of the above-described components communicates data through a data communication bus.

The computer system may further include a network interface coupled to the network. A processor may be a central processing unit (CPU) or a semiconductor device that processes instructions stored in memory and/or storage.

Memory and storage may include various types of volatile or non-volatile storage media. For example, memory may include ROM and RAM.

Therefore, the method for determining readiness for switching control of autonomous driving according to an embodiment of the present invention can be implemented in a method executable on a computer. When the method for determining readiness for switching control of autonomous driving according to an embodiment of the present invention is performed on a computer device, computer-readable instructions may perform the method of determining readiness for switching control of autonomous driving according to the present invention. .

Meanwhile, the method for determining readiness for switching control of autonomous driving according to the present invention described above can be implemented as computer-readable code on a computer-readable recording medium. Computer-readable recording media include all types of recording media storing data that can be deciphered by a computer system. For example, there may be read only memory (ROM), random access memory (RAM), magnetic tape, magnetic disk, flash memory, and optical data storage devices. Additionally, the computer-readable recording medium can be distributed to a computer system connected through a computer communication network, and stored and executed as a code that can be read in a distributed manner.

Claims (19)

A recognition unit that acquires status information of the driver inside the vehicle and obtains driving-related information of the vehicle;
a determination unit that calculates a safe driving capability index of the driver using the driver's status information and driving-related information, and determines whether driving control rights can be switched based on the safe driving capability index; and
A control unit that generates a warning according to the safe driving possibility index, monitors changes in the driver's status following the occurrence of the warning, and transmits a control command according to a preset risk minimization strategy,
The determination unit determines whether the driving posture is normal using the handle grip position, handle grip pressure, position of both feet, seat back angle, seat cushion angle, and seat position information, and determines whether the driving posture is normal, such as leaning forward, driving with one hand, or placing one's legs on the seat. Depending on the driving posture category, including driving posture, straight posture, drinking, checking rear seat passengers, operating devices, poor grip of both hands on the steering wheel, and seat folding, it checks whether the above-mentioned normal driving posture is present and performs pre-learning. Compare the normal state driving posture data and the current driver's state information, and determine whether at least one of the minimum, maximum, and average values of the safe driving index is included within the data category of the normal state in which safe driving is possible. Thus, it is determined whether the driving control right can be switched,
Following the occurrence of the warning provided by the control unit, the determination unit updates the safe driving capability index using the driver's status change information obtained by the recognition unit, and the updated safe driving capability index is preset. If the level remains below the value for more than a preset time, a control request is sent according to the risk minimization strategy,
When receiving a control request according to the risk minimization strategy, the control unit transmits a control command for stopping and parking the vehicle on the shoulder in consideration of the driving-related information including driving path information and the map construction state,
Even though the control unit provides a driver warning, if the consistency with the normal driving posture model using the current video, steering information, braking information, and seat-related information remains below a certain value for more than a previously set time, the control unit Implementing controls in accordance with the above risk minimization strategy without performing transitions.
A device for determining readiness for switching control of autonomous driving. delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete