CN106945672B

CN106945672B - Method and controller for outputting drowsiness warning

Info

Publication number: CN106945672B
Application number: CN201611168367.XA
Authority: CN
Inventors: R.博斯
Original assignee: Robert Bosch GmbH
Current assignee: Robert Bosch GmbH
Priority date: 2015-12-16
Filing date: 2016-12-16
Publication date: 2021-11-26
Anticipated expiration: 2036-12-16
Also published as: CN106945672A; US9984298B2; US20170177959A1; DE102016205996A1

Abstract

The present invention relates to a method for outputting a drowsiness warning. First, the drowsiness (114) of the driver (106) of the vehicle (100) is read. A display signal (118) is then generated based on the drowsiness level (114). The display signal (118) comprises an image ordered set (116) as the drowsiness warning, which image ordered set is displayable by a display (110) of the vehicle (100) and comprises at least one eye part (108) of the driver (106).

Description

Method and controller for outputting drowsiness warning

Technical Field

The invention relates to a device or a method according to the type of the independent claims. The subject of the invention is also a computer program.

Background

It is of great interest to be able to recognize or predict drowsiness of the driver of a vehicle. Fatigue significantly limits driving suitability to a similar extent as alcohol. If a person is tired, he will be harder to focus, react more slowly and evaluate himself too high. This may adversely affect the execution of driving.

Disclosure of Invention

Against this background, the approach presented here is used to describe the method for outputting drowsiness warnings according to the independent claims, as well as the controller used for the method and finally the corresponding computer program. The device indicated in the independent claim can be improved and improved by the measures cited in the dependent claims.

A method for outputting a drowsiness warning is presented, wherein the method comprises the steps of:

reading the drowsiness degree of a driver of the vehicle; and generating a display signal using the drowsiness, wherein the display signal comprises an image sequence as a warning of the drowsiness, which image sequence can be displayed by a display of the vehicle and comprises at least one eye part of the driver.

The drowsiness warning can be output, for example, to the driver, another vehicle occupant, or a third person via a suitable display. The display can be an appliance integrated into the vehicle or arranged in the vehicle, for example a mobile telephone of a passenger of the vehicle.

The drowsiness can be read in the form of a signal or digital data via a read-in interface. The drowsiness can for example be understood as a drowsiness index which shows the drowsiness of the driver. The drowsiness can be obtained, for example, based on the eye-closing speed, eye-opening level, eyelid-closing duration, or specific eye movements of the driver. The Display can be understood, for example, as a central Display, a cluster or a Head-up Display (Head-up-Display) of the vehicle. In this case, the display can be seen by the driver of the vehicle or by at least one further passenger of the vehicle, depending on the embodiment. An image sequence is understood to mean a number of images that follow one another in time, in particular a video sequence. The eye part can be understood as the face of the driver comprising the driver's eyes.

The solution described here is based on the following recognition: an image sequence, for example a video sequence, can be used to warn the driver, for example in the context of video-based drowsiness recognition or snooze recognition. By means of the display of the video sequence, the driver can notice himself, for example in order to see: how long it closes the eyes, how slowly its eyelids move, or how tired it looks. This has the following advantages: the driver is more clearly aware of the risks associated with his drowsiness and therefore tends earlier to insert a driving break. Therefore, the driver can be effectively prompted to perform driving interruption by such a warning design, so that the driver can safely reach the destination and accidents are reduced.

The method can include the steps of: the image sequence (bildfold) is selected from a sequence of images (Bildsequenz) representing a sequence captured by an observing driver camera device. The observation driver camera device can be understood here as an image recording device having a recording range in which the eye region of the driver is recorded during driving. The image sequence can be an actually acquired or stored image sequence. The method can thus comprise the following steps: the image sequence is read in via an interface to the observing driver camera device or a storage means. Furthermore, the method can further comprise the steps of: a sequence of images including at least a part of the eye is recorded. For example, in the selecting step, the image sequence can be selected from the sequence according to an eye closing speed, an eye opening level, an eyelid closing duration, a turning of the eyes, or a yawning of the driver. A drowsiness warning very close to reality can be achieved using the image sequence.

According to one embodiment, in the step of reading in, a signal provided when using an observation driver camera of the vehicle or a yaw-prevention warning system of the vehicle or a steering angle of the vehicle can be read in as the drowsiness degree. The drowsiness can represent, for example, a signal derived from a photograph of the observing driver camera or the position or steering angle of the vehicle within the lane. By this embodiment a reliable and accurate analysis of the drowsiness of the driver is achieved.

According to another embodiment, the method can comprise the steps of: displaying the image ordered set through the display using the display signal. The image sequence can be displayed to the driver. Alternatively or additionally, the image sequence can be displayed to at least one other occupant of the vehicle, in particular, for example, a passenger. By this embodiment, the image ordered set can be played to the driver or other occupants.

Advantageously, said drowsiness is analyzed in said step of analyzing. The display signal can be generated in the generating step as a function of the result of the analysis. For example if the result yields: the drowsiness indicates a slight sleep of the driver, the display signal can then be generated. Additionally or alternatively, if the result yields: the drowsiness exceeding a predefined limit value, the display signal can be generated. The limit value can be, for example, a specific eyelid closing frequency or a specific eye opening level. This increases the tolerance of the method.

According to another embodiment, the method can comprise the steps of: providing a visual and/or audible and/or tactile warning signal for warning the driver of the drowsiness in dependence of the drowsiness. The effectiveness of the drowsiness warning can thereby be further improved. The method can thus be integrated into a system which detects drowsiness of the driver, for example on the basis of the steering angle of the vehicle or by means of a yaw-proof sensor, or detects it by eye closure and then outputs a corresponding warning. The warning can be inserted into a rest break, for example in the form of a symbol, for example by displaying a coffee cup to urge the driver.

Advantageously, the step of generating is performed immediately following the step of providing in time. The driver can thus be warned of the drowsiness at two different warning levels.

For example if in the step of analyzing: said drowsiness showing a slight sleep or exceeding said limit value, said warning signal can be provided in said step of providing. The step of reading in can be carried out again immediately after the step of providing, in order to read in further drowsiness of the driver. Accordingly, the step of analyzing can be re-performed in order to analyze the other drowsiness. Finally, if, in the step of analyzing, it is found that: said other degree of drowsiness indicating said microsleep or exceeding said limit value, said display signal can be generated in said generating step.

The method can be implemented, for example, in the form of software or hardware or in the form of a mixture of software and hardware, for example, in a controller.

In addition, the embodiments described herein provide a control unit which is designed to implement, control or carry out the steps of the variants of the method described herein in a corresponding device. Furthermore, the object on which the invention is based can be solved quickly and efficiently by means of a variant of the invention in the form of a controller. Such a control device can in particular have a read-in unit for reading in the drowsiness of the driver and a generation unit for generating the display signal in the case of use of the drowsiness.

For this purpose, the controller can have at least one computation unit for processing signals or data, at least one memory unit for storing signals or data, at least one interface to sensors or actuators for reading in sensor signals from the sensors or for outputting control signals to the actuators, and/or at least one communication interface for reading in or outputting data, the computation unit, the memory unit, the interface and/or the communication interface being embedded in a communication protocol. The computing unit can be, for example, a signal processor, a microcontroller or a similar appliance, wherein the memory unit can be a flash memory, an EPROM or a magnetic memory unit. The communication interface can be designed to read in or output data wirelessly and/or by wire, wherein the communication interface, which can read in or output wired data, can read in or output this data from or into the respective data transmission line, for example electrically or optically.

In an advantageous embodiment, the control of a central display, a cluster or a head-up display of the vehicle is effected by the control unit. For this purpose, the controller can access sensor signals, for example, which observe the driver camera, the steering angle sensor or the yaw-proof warning system.

It is also advantageous to provide a computer program or a computer program product having a program code, which can be stored on a machine-readable carrier or storage medium, such as a semiconductor memory, a hard disk memory or an optical memory, and is used to implement, realize and/or handle the steps of the method according to one of the embodiments described above, in particular when the program product or program is executed on a computer or a device.

Drawings

Embodiments of the invention are illustrated in the drawings and are further described in the following description. Wherein:

FIG. 1 shows a schematic diagram of a vehicle having a controller according to one embodiment;

FIG. 2 shows a schematic diagram of a controller according to an embodiment;

FIG. 3 shows a block diagram of a method according to an embodiment; and is

FIG. 4 shows a flow diagram of a method according to an embodiment.

In the following description of advantageous embodiments of the invention, the same or similar reference numbers are used for elements which are shown in different figures and which function similarly, and a repeated description of these elements is omitted here.

Detailed Description

FIG. 1 shows a schematic diagram of a vehicle 100 having a controller 102 according to one embodiment. The vehicle 100 has an observation driver camera device 104, the observation driver camera device 104 being used to capture the driver 106 of the vehicle 100, in particular the eye region 108 of the driver 106, and a display 110, the display 110 being used, for example, to display information in the field of view of the driver 106. The display 110 is, for example, a central display, a combination meter or a head-up display, or else a mobile appliance, such as, for example, a smartphone, which is arranged in the vehicle 100. The controller 102 is connected or coupled to not only the observing driver camera device 104 but also the display 110. Furthermore, the controller 102 is connected to a drowsiness detection unit 112, which is designed to detect a drowsiness level 114 that represents a drowsiness of the driver 106 and to send a corresponding signal to the controller 102. According to an embodiment, the drowsiness acquisition unit 112 is configured for acquiring the drowsiness 114 using a signal provided by the observing driver camera device 104 or by a yaw-proof warning system of the vehicle 100 or based on a steering angle of the vehicle 100. As exemplarily shown in fig. 1, the drowsiness acquisition unit 112 can be implemented as a unit separate from the controller 102. Alternatively, the drowsiness acquisition unit 112 can also be implemented as a component of the controller 102 or a component of the observation driver camera device 104.

The observing driver camera device 104 is configured to capture an image sequence 116 of the eye part 108 and to transmit the image sequence 116 to the controller 102. The controller 102 is configured to generate a display signal 118, e.g., including the image ordered set 116, based on the drowsiness 114. The display 110 is designed to display the image sequence 116 using the display signal 118, so that the driver 106 can notice himself in the display 110.

For example, if the degree of drowsiness 114 is analyzed by the controller 102: the drowsiness of the driver 106 suggests a nap or the drowsiness 114 exceeds a predefined limit, the display signal 118 is generated by the controller 102.

According to an embodiment, the observing driver imaging device 104 captures the eye part 108 continuously or at certain intervals. The observing driver camera device 104 is configured, for example, to capture a continuous sequence of images having a series of single images and transmit the sequence of images to the controller 102. Accordingly, the controller 102 can be configured to select the image ordered set 116 from the sequence of images based on the drowsiness 114.

According to this embodiment, the controller 102 is configured to generate an optional warning signal 120, here an audible warning signal, in addition to the display signal 118, also as a function of the drowsiness 114, and to transmit the optional warning signal 120 to a corresponding playback unit 122, here to a loudspeaker in the vehicle 100. The playback unit 122 is designed to output a warning, for example a warning sound or a warning announcement, to the driver 106 in the event of the use of the warning signal 120, in order to warn the driver 106 of drowsiness. Depending on the embodiment, the warning signal 120 can also be a visual or tactile warning signal or a combination of different warning signals. For example, the following warning possibilities can be envisaged. The driver 106 can be warned, for example, by visual feedback in the form of a status display coded in terms of color (also referred to as an alarm meter), a coffee cup, or a warning triangle. In addition or alternatively, it is also possible to warn by tactile feedback, for example by a vibration of a safety belt, by a pressure pulse acting on the palm of the driver 106 or by a steering wheel vibration or a seat vibration. It is also conceivable that the olfactory warning is carried out by a suitable means for transmitting the scent, for example by transmitting a peppermint scent.

According to one embodiment, a warning to the driver 106 is triggered based on the output signal of the observing driver camera device 104 by the driver function module "drowsiness and/or nap". Alternatively, the output of drowsiness detection (english) is used based on the steering angle of the vehicle 100 or the tracking error signal of the yaw prevention warning system. If a nap is identified, or additionally or alternatively the drowsiness exceeds a certain threshold, a previous video sequence, for example, is displayed to the driver 106 or another occupant of the vehicle via the display 110. If the previous video sequence cannot be accessed, for example, the most recent event possible with the drowsiness-causing feature or in the simplest case the next video sequence immediately following it is presented. The characteristics caused by drowsiness can be for example: slow eye closure speed, low eye opening level, eyelid closure duration, eye turning or yawning. The video sequence can include the entire face of the driver 106 or only the eye part 108, depending on the embodiment. Alternatively, before displaying the video sequence, a typical warning in the form of a warning signal 120, for example a warning symbol in the shape of a coffee cup or an audible warning, is output to the driver 106 either via the playback unit 122 or via the display 110 or via both devices. If the driver 106 thereafter does not introduce countermeasures, i.e. no break pause or no driver change, the video sequence can be displayed to the driver 106 as a second warning level after the warning signal 120 has been played. A schematic flow is described below with the aid of fig. 3.

FIG. 2 shows a schematic diagram of the controller 102 according to an embodiment. The controller 102 is, for example, the controller described above with the aid of fig. 1. The controller 102 comprises a read-in unit 200 for reading in the drowsiness 114 and a generation unit 202 for generating the display signal 118 in dependence of the drowsiness 114. The

units

200, 202 of the controller 102 are designed, for example, to carry out the steps of the method described with reference to fig. 4.

Fig. 3 shows a block diagram of a method for outputting a drowsiness warning, for example, to the driver 106 of a vehicle, according to an embodiment. The driver 106, the observing driver camera device 104 and a computing unit as a controller 102 (as already described for the controller with the aid of fig. 1) are schematically illustrated, the observing driver camera device 104 being configured to capture the face of the driver 106, at least the eye part 108. The controller 102 is configured to obtain in step 300: whether the driver 106 is asleep, and in step 302: whether the drowsiness index (also referred to as drowsiness in the foregoing) associated with the driver 106 exceeds a predefined threshold. If the driver 106 is asleep or the drowsiness index is above the threshold, a video sequence having drowsiness characteristics indicative of a slight sleep is displayed on the display, e.g., the central display of the vehicle, a cluster or a head-up display, in step 304. A visual, audible or tactile warning is additionally output in an optional step 306, which precedes step 304.

FIG. 4 shows a flow diagram of a method 400 for outputting a drowsiness warning, according to one embodiment. The method 400 can be implemented, for example, by the controller described above with the aid of fig. 1 to 3. In this case, the drowsiness of the driver is read in step 410. In a further step 420, a display signal is generated as a function of the drowsiness level for warning the driver or another vehicle occupant against the drowsiness of the driver. The display signal is displayable to the driver and/or another vehicle occupant by a display of the vehicle and comprises an image ordered set presenting at least one eye part of the driver as the drowsiness warning.

If the image order is not already present, the image order can be selected from the sequence of images in the generating step 420. According to one specific embodiment, in step 420, the image sequence is first read in from a memory device or via an interface to an observation driver camera device and represents an image sequence of the eye part of the driver, which is actually captured by the observation driver camera device or captured temporally first.

According to an embodiment, the method 400 comprises an optional step 430, in which optional step 430 the image ordered set is displayed by the display. According to an exemplary embodiment, the image sequence is displayed to the driver of the vehicle or to another passenger of the vehicle, for example a co-driver.

According to another embodiment, the drowsiness is analyzed in an optional step 440 following the read-in. Here, if in step 440: if the drowsiness indicates a slight sleep of the driver or, in addition or alternatively, exceeds a predefined limit value, the display signal is generated in step 420.

If an embodiment includes the conjunction "and/or" between a first feature and a second feature, then it may be interpreted as follows: the exemplary embodiment has both the first and the second features according to one specific embodiment, and either only the first or only the second feature according to another specific embodiment.

Claims

1. Method (400) for outputting a drowsiness warning, wherein the method (400) comprises the steps of:

reading (410) a drowsiness (114) of a driver (106) of a vehicle (100); and is

-generating (420) a display signal (118) using the drowsiness degree (114), wherein the display signal (118) comprises an image sequence (116) as the drowsiness warning, which image sequence can be displayed by a display (110) and comprises at least one eye part (108) of the driver (106), wherein in the step of generating (420), the image sequence (116) is selected from an image sequence according to the drowsiness degree, which image sequence represents an image sequence of the eye part (108) of the driver (106), which image sequence is captured by an observing driver camera device (104), the method having the following steps: analyzing (440) said drowsiness (114), wherein in said step of generating (420) said display signal (118) is generated based on the result of said analyzing (440), if said result yields: the drowsiness exceeding a predefined limit value can then generate the display signal, a typical warning signal is output to the driver via a playback unit and/or a display before the image sequence is displayed, and the image sequence is displayed as a second warning level to the driver after the warning signal has been played if the driver has not introduced countermeasures thereafter.

2. The method (400) according to claim 1, wherein in said step of reading in (410) a signal provided using an observed driver camera device (104) of said vehicle (100) and/or a yaw-proof warning system of said vehicle (100) and/or a steering angle of said vehicle (100) is read in as said drowsiness degree (114).

3. The method (400) according to claim 1 or 2, having the step of displaying (430) the image ordered set (116) by means of the display (110) using the display signal (118).

4. The method (400) of claim 1 or 2, having the steps of: providing a visual and/or audible and/or tactile warning signal (120) in dependence of the drowsiness level (114) for warning the driver (106) of the drowsiness.

5. The method (400) of claim 4 wherein said step of generating (420) is performed immediately following said step of providing in time.

6. A controller (102) having a unit (200, 202) configured for performing and/or handling the method (400) according to any of the preceding claims.

7. A machine-readable storage medium, on which a computer program is stored, which is designed to carry out and/or to control the method (400) according to any one of claims 1 to 5.