CN116529143A

CN116529143A - Communication system for a vehicle for behavior in the event of sleep disturbance of an occupant

Info

Publication number: CN116529143A
Application number: CN202180073772.6A
Authority: CN
Inventors: V·贝德
Original assignee: Volkswagen AG
Current assignee: Volkswagen AG
Priority date: 2020-11-19
Filing date: 2021-10-28
Publication date: 2023-08-01
Also published as: DE102020214556A1; WO2022106176A1

Abstract

A method for operating a communication system (20) for a vehicle (1), having at least one input device (22) for inputting at least one command for at least one vehicle function of the vehicle (1) by an occupant (10) of the vehicle (1), wherein the communication system is adapted and determined to cause the implementation of the command by providing an implementation signal in response to the input. According to the invention, a state of consciousness, in particular a sleep state, of the occupant (10) is detected and/or determined when the command is entered, and the communication system (20) evaluates on the basis of the state of consciousness, in particular the sleep state, whether it is an input caused by the state of consciousness, in particular the sleep state, and/or by a neurological disorder, in particular a sleep disorder.

Description

Communication system for a vehicle for behavior in the event of sleep disturbance of an occupant

Technical Field

The present invention relates to a method for operating a communication system of a (motor) vehicle and also to a communication system for a (motor) vehicle. The communication system has at least one input device, such as, for example, a man-machine interface, for inputting at least one command by an occupant of the vehicle, which command is associated with at least one vehicle function of the vehicle. In particular, the present invention relates to taking into account the input of an occupant during a sleep state, in particular the input caused by a sleep disturbance of the occupant. In particular, the invention relates to behavior in situations where a vehicle occupant is in dream, moving during sleep, or speaking during sleep.

Background

Vehicles are currently commonly equipped with a communication system that is adapted and determined to be implemented by providing an implementation signal such that the command corresponds to the input. Thus, the implementation of the instructions can be realized and, for example, the driving function can be changed, activated or a function setting can be performed.

Methods are known from the prior art in which the sleep state of a vehicle occupant is detected, for example, in the event of fatigue, the vehicle driver is warned or the driving comfort of the vehicle occupant is increased.

In WO 2018/226220 A1 a system is described in which a sleep state of a vehicle occupant is identified. For this purpose, the vehicle movement is compared with the passenger movement. Based on the identified sleep state and vehicle position, activation of a preset vehicle system may be performed. In addition, a voice recognition system is provided, in combination with a system for acquiring facial expressions or lip movements, in order to recognize the awake state of the occupant based thereon.

From WO 2019/122114 A1 a vehicle control system with physiological motion recognition (like e.g. gestures or breathing) is described, for example input control for an entertainment system, a navigation system or a system for automatic driving of a vehicle. Thus, for example, a sleep state can be derived. Based on the physiological motion signal, the vehicle control system may indicate, via a speaker, sound output, setting of glass transparency, matching of lighting conditions, movement of the vehicle.

In DE 10 2016 225 222 A1 a method is described for influencing the interaction process between vehicle equipment (e.g. for the output of media content or internet access) and vehicle occupants in accordance with parameters which are characteristic for the physiological or psychological condition of the occupants, furthermore fatigue, health, sadness, happiness. If the user is tired, the brightness of the vehicle interior space can be reduced.

US 2020/013992 A1 describes a control method for a vehicle in which the reactivity and wakefulness of the driver are acquired. These states are used to switch between a mode in which the vehicle is autonomously controlled and a manual driving mode of the vehicle.

An infotainment system for a motor vehicle is known from DE 10 2016 213 916 A1. This may include VR glasses, where sleep recognition may be integrated. This makes it possible to take into account, for example, that the user wants to listen to music with his eyes closed and that the transmission of the movie is stopped when a preset period of time is exceeded. For sleep recognition, electrical signals from brain waves, heart beat, muscle tone, and movement and respiratory sounds (e.g., snoring or other sound expressions) during sleep may be considered. The wake-up of the driver may be achieved taking into account the determined sleep stage.

Especially with regard to fully automatic driving vehicles ("class 5"), there are furthermore more and more user scenarios in which the vehicle occupants can relax or even fall asleep in the cabin. In particular, even usage scenarios are designed in which a vehicle occupant comfortably brings a vehicle from a starting point (e.g., a door) to a destination (e.g., a vacation destination) in a fully automated (autonomous) driving, for example, overnight, and is able to fall asleep comfortably during automated driving. In DE 10 2019 204 201.5, it is proposed to adapt the driving pattern of an autonomous vehicle to the sleeping state of the vehicle occupants. Furthermore, DE 10 2020 202 388.3 describes a behavior when a vehicle occupant wakes up in an autonomous vehicle during driving. DE 10 2017 220 120 describes a nerve feedback system which should be made easy in the cabin until falling asleep, and in which fatigue and sleep states can be identified.

However, during sleep time, many people may also have a state in which they "speak during sleep" (so-called dream), unconsciously perform active exercise, or even perform actions from daily life in an unconscious state in a state of "dream" (dream disorder). This state of the vehicle occupants can thus also occur in the use scenario of the autonomous vehicle described above, in which people are transported in the cabin during sleep times.

In the case of sleep disturbance of dream (or movement in sleep), the person affected is caused to wake partly from deep sleep. The affected person in this state has less consciousness, but will perform operations that are normally performed only in the awake state. These are generally simple, repeatedly performed movements, such as sitting up in a bed, chewing movements or pulling a garment. However, in addition, complex actions such as walking around, cleaning or cooking are also possible. In particular, the eyes of the affected person are open, while their facial expression is fixed. In addition, the affected person has little response to environmental stimuli and is very difficult to wake up. Sometimes, noise is generated when the affected person speaks. The affected person cannot remember this fragment after waking up. In most cases, dreaming occurs in the first third of the evening, where most deep sleep also occurs.

About 10-30% of children and adolescents have at least one stage of nocturnal emission, and 15% of children and adolescents are often affected by sleep disturbance in nocturnal emission. Boys and girls are equally affected. Dreaming typically begins between four and eight years of age. Sleep disorders of dreaminess are usually resolved by themselves during puberty. The dream may be associated with other disorders that occur during sleep, such as nocturnal frightening (pavor nocrturnus) (nocturnal frightening-a frightening that is strongly excited during deep sleep stages without actually and consciously waking up), restless leg syndrome, sleep disordered breathing and/or nocturia.

From the following, 7% of adults are at least occasionally affected by dream, and 1% to 2% of adults are affected by chronic dream. Disorders in adults with chronic dream are in most cases sustained more and occur more frequently than in children. Furthermore, a sleeping medical clinic reports (taken from https:// www.ksm.ch/de/schlafkrankheiten/pamomnie/symptome-und-assam chen-im-detail) that adult nocturnals are more prone to aggressive and violent behaviour (until the same bed is injured), with the result that they wake up and then have to pay attention to coma.

Although in most cases dream, especially in the case of children and adolescents, is harmless and will disappear by itself before puberty, four precautions are recommended, for example by a sleeping medical clinic in swiss. Such an affected person may be locked with an object that they would injure himself or others, and thus not be able to be taken by the affected person. In addition, the window should also be tight and not too easy to open. For example, a bell should also be installed on the bedroom door, which reports when it leaves the room. Finally, the new environment should be informed of the existence of sleep disturbances of the dream and observe precautions there as well, since the risk of accidents is particularly high in unfamiliar and unfamiliar environments.

Another type of abnormal sleep, the so-called speaking dream, refers to speaking during sleep, ranging from reproducing undifferentiated sounds to words and whole sentences. It is said that dreaminess occurs in about 50% of children and about 5% of adults (taken from https:// derschlaffuchs. De/student-reden-im-schlaf-synthax-und-semmantik-von-schlafbezogenen-aeusservengen/; see also Arnulf, isabelle et al, "What Does the Sleeping Brain SaySyntax and Semantics of Sleep Talking in Healthy Subjects and in Parasomnia Patients", sleep, vol.40, no.11,2017, doi:10.1093/Sleep/zsx 159). Typically speaking during sleep occurs when alternating between different phases of non-REM sleep, but may also occur during REM sleep, so that phrases in dreams are spoken aloud.

Sleep disturbances such as sleepiness or sleepiness may affect transportation in an automated vehicle. For example, in a car, sleep disorders such as "dream", exercise during sleep, or "talk during sleep" may have the effect that a person in these states may unintentionally activate and/or deactivate vehicle functions, or, for example, change navigation destinations by speaking alone. The more interactions between the vehicle and the vehicle user based on gestures and/or speech, the more important this becomes.

In the methods for operating a communication system and in communication systems for vehicles known from the prior art, the disadvantage arises that the input of the occupant triggered by a sleep disturbance can lead to dangerous situations of the occupant himself or of other occupants.

Disclosure of Invention

The present invention is based on the task of overcoming the disadvantages known from the prior art and providing a method for operating a communication system and a communication system which prevent dangerous situations of the occupants due to inputs from sleep disorders of the occupants and at the same time ensure a high reliability in the operation of the vehicle functions by means of inputs from the occupants.

According to the invention, this object is achieved by the subject matter of the independent claims. Advantageous embodiments and improvements of the invention are the subject matter of the dependent claims.

In the method according to the invention for operating a communication system for a (motor) vehicle, in particular for taking into account the sleep state of an occupant, the communication system has at least one input device for inputting at least one command by the occupant of the vehicle for at least one vehicle function of the vehicle. The communication system is here adapted and configured to cause the instruction to be executed on the basis of the input by providing an execution signal. Preferably, the vehicle is an at least partially autonomously operable vehicle.

In the method according to the invention, a state of consciousness, in particular a sleep state, of the occupant is detected and/or determined, in particular when a command is input, and the communication system evaluates on the basis of the state of consciousness, in particular the sleep state, whether it is an input by the state of consciousness, in particular the sleep state, and/or a neurological disorder, in particular a sleep disorder. In particular, it is a state of consciousness, in particular a sleep state, of the occupant at the point in time of the input. The neurological disorder may in particular be a disorder caused or triggered by a disorder of the nervous system. For example, the neurological disorder may be a sleep disorder. It is also possible that the neurological disorder is a (especially short-term) dysfunction and/or a (neurological) disorder of the brain, such as e.g. epilepsy.

The state of consciousness of the occupant can be understood here as a (particularly conscious) state of the occupant, in which for example a conscious state or a conscious state (alertness) is possible for the occupant, a perception in the sense of a conscious experience (e.g. sensory perception, action intention). A conscious state may also be understood as a state in which the occupant is not awake. Furthermore, a state of consciousness may be understood as a sleep state and/or a state of distraction.

The invention is described below with respect to the acquisition and/or determination of sleep states (as a state of consciousness) and sleep disorders as neurological disorders. However, features described below in connection with sleep states and/or sleep disorders may also be transferred in an equivalent manner to a state of consciousness or neurological disorder, and should also be considered disclosed in this regard.

Sleep disorders can be, in particular, movements and/or dream during sleep and/or dream (speech during sleep).

Based on this evaluation, the communication system can, for example, (preferably for a state of consciousness, in particular a sleep state and/or a preset duration of sleep stages, and/or for a duration of a segment of a sleep disturbance) cause (in particular temporarily) a blocking of the input possibilities and/or change of the input mode. In this case, it is preferably (only) a blocking of the input possibilities and/or a change of the input modes for the passenger and in particular not for another passenger. The change of the input modality can be understood, for example, as an additional requirement for (further) input confirmation by the occupant.

In other words, the communication system checks whether the occupant is in a sleep state at the input time point. Starting from this, the communication system can evaluate whether the passenger has not performed his input consciously, since he is, for example, in a sleep state and/or a defined (subsequently implemented) sleep phase and is not able to perform conscious actions, in particular is not able to perform conscious inputs, in this sleep state and/or in this sleep phase.

Preferably (by the communication system) a probability value is determined, which characterizes the probability that a sleep state and/or a (preset) phase is present at a preset point in time and/or time period, and particularly preferably at the point in time when the occupant is taking an input.

The probability value is preferably compared, in particular by the communication system and preferably by at least one processor of the communication system, with a threshold value, in particular preset and/or which can be input to the communication system. The threshold value may be fixedly preset and stored in the memory device. The threshold value may preferably be preset in an occupant-specific manner.

Based on the result of this (or of a plurality of such comparisons performed), it is preferably assessed (in particular by the communication system) whether the occupant is in a conscious state and/or a sleep state, in particular at the time of the input, in which the occupant does not (fully) consciously decide on his/her behavior.

In the evaluation, it is preferably taken into account whether at least one or another feature is present and/or an indication of the presence of a sleep disturbance, in particular at the time point of the input. The corresponding features may preferably be considered in connection with the presence of certain sleep stages (explained in more detail below), such as, for example, REM sleep, non-REM sleep and/or transitions between two sleep stages and/or transitions between different stages of non-REM sleep. This allows for consideration of the features that typically occur in determining sleep stages.

Such features may be common side effects of sleep disturbance acquired and/or determined (e.g., by the acquisition device and/or the additional acquisition device). For example, in such side effects, as mentioned at the outset, sleep disorders of the dream are, for example, another sleep disorder such as nocturnal convulsions, restless leg syndrome, sleep-in-respiratory disorders and nocturia.

Features to be considered in the evaluation, especially in sleep disturbance of a speaking dream, may be to obtain indistinguishable sounds and/or non-verbal expressions (whisper, shouting, whisper, laughing) and/or sporadic words (which the occupant may hear during sleep). Thus, one study found that 59% of the expression of the dream-speaking fragment was non-linguistic (see Arnulf, isable et al, "What Does the Sleeping Brain SaySyntax and Semantics of Sleep Talking in Healthy Subjects and in Parasomnia Patients", sleep, vol.40, no.11,2017, doi:10.1093/Sleep/zsx 159). The characteristic may preferably be a ratio of non-verbal expression to verbal expression of the occupant, which characterizes in particular the presence of a segment of the spoken dream.

The feature may preferably be a frequency of the preset word. The above study can determine that "no" is the most common word and that the negative occurrence in sentences is 21.4% (more frequently in non-REM sleep). The feature may preferably be a feature variable for frequency, determining sentence structure (question, clause). The feature may preferably be a feature variable for semantic analysis of the linguistic expression of the occupant, such as using visceral speech (more frequently in non-REM sleep), speech abuse, counter-productive, judgment, curse. The above study shows that not less than 9.7% of sentences speaking the segment of dream contain visceral speech.

Here, the one or more features may be general features or occupant-specific features. For example, in the case of a known dream of an occupant, an evaluation of the syntax or semantics of the occupant in the segment that has occurred before can be considered in the features. Such features and/or data may be stored in a user profile.

The indication may be a risk estimate for sleep disturbance occurrence. Such an indicator may be the age of the occupant and/or the frequency with which the occupant has developed a sleep disorder (during a previous period of time).

In this way, the intentional and intentional input by the occupant can advantageously be distinguished from the unintentional input by the occupant, and thus an accidental input of instructions can be avoided.

The instructions relating to the vehicle function are to be understood as, in particular, the activation, deactivation and/or operation of the vehicle function, a change in the function settings relating to the vehicle function and/or a preset and/or change in the data input for the vehicle function.

The vehicle function may be a function of a vehicle component of the vehicle. The vehicle component is preferably selected from the group comprising a locking system for doors and/or windows of the vehicle, a vehicle closure system, a roof removal mechanism, a sunroof mechanism, an infotainment system, an entertainment system, a navigation system, a vehicle safety system (e.g. a braking system), a windshield wiper mechanism, a system for driving the vehicle and/or a comfort system for increasing the driving comfort of the occupants, and combinations thereof.

The input device may be a gesture and/or facial expression and/or voice based input device, an activity tracking means for detecting movements of an occupant (e.g. wrist), a tactile operating element (e.g. switch, joystick), a touch sensitive operating element, e.g. touch sensitive display, a graphical based input device by means of a visual interaction element, a physiological state based determination input device and/or a perceptually controlled input device.

The input device may preferably be an interactive device. The input by the occupant may be an interaction between the occupant and the input device.

For a preferred activity tracking device, reference is made in particular to publication DE 10 2015 120 803A1, the content of which is hereby incorporated into the present application. An activity tracking device is described therein that detects movement of an occupant's wrist and is preferably capable of exchanging this data with a communication system and/or a vehicle data processing system. The communication system and/or the vehicle data processing system preferably generates at least one output signal based on the data of the activity tracking device. For example, the control device may operate a windshield wiper mechanism or a sunroof mechanism based on a signal indicating the detected long wiping movement.

For a suitable and determined preferred gesture-based input device for gesture-based interaction between a vehicle and a user or occupant, reference is made in particular to application DE 10 2016 121 032 A1, the content of which is hereby incorporated into the present application. The input device described therein enables the implementation of vehicle functions to be imported by gestures.

For a preferred input device based on the determination of the physiological state, reference is made in particular to publication DE 10 2017 101 343A1, the content of which is hereby incorporated into the present application. Physiological data of the vehicle occupants may be acquired via the input device and may be used to determine one or more physiological states of one or more vehicle occupants, in order to control the vehicle therewith, preferably.

With respect to a preferred method for monitoring the vigilance state of a person by means of a sensor device for acquiring at least two body-expressed features of the person, wherein at least one body-expressed feature is a non-eye-based body-expressed feature, reference is made in particular to publication DE 10 2017 216 328 B3, the content of which is hereby incorporated into the present application.

For a preferred input device based on facial expressions and/or gestures, reference is made in particular to publication DE 10 2017 216 837A1, the content of which is hereby incorporated into the present application. Such an input device preferably has a plurality of devices and a human-machine interface (HMI) for mimicking and/or gesture-manipulating the functions of the vehicle device. A camera for detecting a passenger of the vehicle and a control unit, in particular connected to the camera, are preferably provided.

For a gesture-based preferred input device, reference is made in particular to publication DE 10 2018208 866a1, the content of which is hereby incorporated into the present application. A method of recognizing human input by means of gestures is described therein.

For a preferred input device based on pointing gestures, reference is made in particular to publication DE 102018211 624a1, the content of which is hereby incorporated into the present application. A method and user interface for acquiring input by means of a pointing gesture is described therein.

For a preferred speech-based input device, reference is made in particular to publication DE 10 2019119 171A1, the content of which is hereby incorporated into the present application. Wherein speech recognition is provided for vehicle voice commands.

The input device may be part of the vehicle equipment and may in particular be arranged firmly (non-destructively detachable) at the vehicle. It is also conceivable that the input device may be a mobile terminal device (e.g. a smart phone, a tablet or a notebook for example for the occupants), which is removable from the vehicle and which may be connected to the communication system, in particular by a wireless and/or wired data communication connection.

In a preferred method, a state of consciousness, in particular a sleep state, and/or an evaluation of whether it is an input caused by a state of consciousness, in particular a sleep state, and/or a neurological disorder, in particular a sleep disorder, is considered in the implementation of the instruction, in particular by modifying and/or preventing the implementation and/or delaying the implementation.

In other words, the acquired and/or determined sleep state of the occupant (when entered) may be considered in the evaluation and/or determination as to (and in what manner) cause the instructions related to the vehicle function. In this case, in particular, it is the sleep state of the occupant (essentially) at the time point of the input.

This provides the advantage that the risk of inadvertent command input by the occupant and thus inadvertent introduction of vehicle functions is significantly minimized.

In particular, it is proposed by the invention that the sleep state and/or the presence of sleep disturbances of a vehicle occupant can be detected and then input possibilities for the vehicle occupant (as a function of the sleep state) for example for gestures or speech recognition and/or active operation of the vehicle function or input possibilities via switches, levers, displays or the like are (in particular temporarily) prevented and/or delayed. Which preferably extends over the entire sleep duration of the occupant and/or over the entire duration of the segment of the speech disturbance.

For example, if a gesture, facial expression and/or voice-based interaction is provided between the vehicle and the occupant or user, in order to activate a function such as a windshield wiper, for example, with a gesture, such interaction may be undesirably carried out in addition to a dream, in the case of carrying out a movement in sleep or speaking in sleep.

The proposed method advantageously enables a vehicle occupant of a (in particular autonomous) vehicle to be unable to activate and/or deactivate any vehicle function and/or to be able to change a function setting and/or to be able to preset or change a data input for a function in a sleep state, in particular in a state of dream, exercise in sleep or speaking in sleep. Preferably, touch interaction with the touch screen may be limited.

Regarding disabling vehicle functions, reference is made to publication DE 10 2009 018074, in which a system is described in which the current driving situation is evaluated and if it is not permitted, no call is forwarded to the driver. The content of this publication is hereby included in the present application.

The particularly brief state of consciousness, particularly the sleep state, of the occupant is preferably acquired by the acquisition device, in particular (substantially) when the occupant is input. It is also conceivable that the awake state is acquired and/or determined in addition to or instead of the sleep state of the occupant and that the sleep state of the occupant is preferably determined from the awake state of the occupant.

The state of consciousness, in particular the sleep state, of the occupant is preferably acquired (by the acquisition device) at regular time intervals. However, it is also conceivable to derive and/or determine the sleep state of the occupant at a later and/or earlier and/or at a point in time between two measurement points in time on the basis of at least one acquired sleep state and preferably on the basis of a plurality of acquired sleep states of the occupant.

It is conceivable to determine a predicted state of consciousness, in particular a sleep state, of the occupant on the basis of a predetermined state of consciousness, in particular a sleep state, for example for a point in time of an input of the occupant, in particular via the acquisition device and/or the communication system.

The acquisition device may be a vehicle interior acquisition device. However, it is also possible that it is an external access device, in particular a mobile access device and/or an access device movable relative to the vehicle, which is connected or connectable wirelessly and/or by wire to the communication system of the vehicle for data exchange.

However, it is also conceivable that the acquired sleep state of the occupant is a signal transmitted to the vehicle and/or the communication system. Here, it may be a preset input of the occupant and/or of another occupant. For example, it may be the input of information about the intentional sleep by the occupant, for example during driving and/or during the determination of the route section.

In a preferred method, the sleep stage and/or fatigue state and/or alertness state of the occupant is acquired and/or determined at the time of input and is taken into account when implementing the instructions. The acquisition device (or a different acquisition device) preferably acquires sleep stages and/or fatigue states. Preferably, the sleep state exists when there is no alert state of the occupant.

The state of consciousness of the occupant is preferably acquired and/or determined based on brain wave measurements (e.g., by an EEG sensor) or based on brain current images (EEG). By means of electroencephalogram measurements (EEG), for example seizures of the occupant caused by epileptic neurological disorders can be acquired.

The sleep state of the occupant and/or the transition from the sleep state to the awake state (and/or vice versa) is acquired and/or determined based on (acquired and/or determined) physiological movements and/or respiratory movements and/or heart beats and/or respiratory frequency and/or limb movements and/or gesture movements and/or body movements and/or brain wave measurements (e.g. by means of an EGG sensor) or brain current images (EEG) and/or heart rhythms (ECG) and/or blood oxygen content (pulse oximeter) and/or body temperature and/or respiratory flow (mouth and/or nose) and/or muscle tension (EMG) and/or leg movements and/or eye movements (EOG) and/or body position and/or body sounds and/or eyelid beats (especially by means of an acquisition device) or a combination thereof.

The acquisition device, in particular for acquiring and/or determining a state of consciousness, in particular a sleep state, and/or a wakefulness and/or a sleepiness of the occupant, preferably has at least one sensor and/or at least one measuring device, which is selected from the group comprising a group of at least one EEG sensor, in particular (directly) measuring brain waves, preferably a plurality of EEG sensors, a microphone, a camera, a motion sensor, an acceleration sensor, a pressure sensor, an eyelid sensor, a respiratory rate sensor, a temperature sensor, a heart rate sensor, a pulse oximeter sensor and combinations thereof.

Preferably, the at least one sensor of the acquisition device and/or at least a part of the acquisition device and preferably a plurality of these sensors may be arranged in a lying area (Liegebereich) of the vehicle and/or in a contact area (for the occupant) of the seat area.

It can be measured here and, if possible, can be deduced from the transition from sleep to awake state from a combination of different sensor means: brain current images (EEG), heart rhythm (ECG), blood oxygen content (pulse oximeter), body temperature, respiratory flow (mouth and/or nose), respiratory movements, muscle tone (EMG), leg movements, eye movements (EOG), body posture, body sounds.

Methods of EEG (electroencephalogram, recording voltage fluctuations on the surface of the head) are known from about 1930 and are used, for example, in sleep laboratories to identify sleep stages or fatigue stages of humans (see, for example, https:// de. Wikipedia. Org/wiki/elektroenzemolografifie).

The acquisition device (especially at least partly and preferably in its entirety) preferably has a sensor for making EEG measurements for acquiring fatigue and/or sleep states and/or conscious states. These sensors are preferably arranged at a pair of spectacles for the occupants. The acquisition device and/or the sensors for making EEG measurements may be arranged here in virtual reality or augmented reality glasses and may preferably be used in a vehicle as such.

The acquisition device, which particularly preferably has a sensor for making EEG measurements (in particular at least partially and preferably in its entirety), is preferably fixedly arranged at the vehicle and in particular at the head support and/or the head region of the lying region. The acquisition device is preferably adapted and determined for performing EEG measurements over a distance (e.g. head-to-head support) so that the alert state and/or fatigue and/or sleep state of the occupant can thus be identified.

With regard to a preferred acquisition device and a preferred method for acquiring and/or determining and/or monitoring the alertness and/or fatigue and/or sleep state of a vehicle occupant (there, the driver), reference is made in particular to publication DE 10 2015 204 247 A1, the content of which is hereby incorporated into the present application. For example, the direction of the line of sight of the driver may be acquired, and whether the driver opens his eyes may be checked. Thus, for example, fatigue recognition is achieved in order to avoid the driver falling asleep in seconds.

The acquisition device preferably has at least one acceleration sensor, in particular an accelerometer, which is preferably positioned in the vicinity of the occupant, for example on a vehicle seat or a vehicle couch, beside or at the occupant. The acquisition device and/or the acceleration sensor are preferably arranged relative to the occupant in such a way that they can record the movement of the occupant. The acceleration sensor may be arranged in a mobile (terminal) device, for example a smart phone. Preferably a sleep stage alarm clock is used as the acquisition device.

Here, the smart phone may be used as an acquisition device together with a so-called "digital sleep stage alarm clock" (a "sleep cycle clock") as a smart phone application (see also https:// de. Wikipedia. Org/wiki/schlabphsase). Such applications typically work as follows: a smartphone with a corresponding application (e.g. "Sleep as Android") is placed next to the desired person in bed. The smartphone then records the movements of the sleeping person using its built-in accelerometer. It should thus and preferably be able to identify which sleep stage the person is in, and whether the person is sleeping, i.e. whether the person is thus in a sleeping state. People experience about three different types of sleep every night: light sleep, deep sleep, and so-called REM ("rapid eye movement") sleep. If who wakes up halfway through the deep or REM phase, it may find it difficult to get up. However, there are so-called "near awake moments" in between, in which it is particularly easy to wake up. These moments are preferably identified by the application.

With regard to a preferred acquisition device and a preferred method for acquiring and/or determining a sleep state of a vehicle occupant, reference is made in particular to publication DE 10 2016 213 916 A1, the contents of which are hereby incorporated into the present application. Described herein are how to support communication and sleep in fully autonomous vehicles in a comfortable manner in connection with the use of virtual reality glasses in the cabin. Here, sleep states are identified and the infotainment and/or virtual reality and/or augmented reality systems in the cabin match the sleep stages of the occupants.

The acquisition device is preferably adapted and determines to analyze the sleep stages of the person or occupant with an analysis of their brain current curves or the like. A suitable wake-up procedure (weckvorg) is preferably placed in the light sleep stage before the last possible wake-up period. Such a solution is known, for example, under the name "SleepSmart" (www.axonlabs.com).

With regard to a preferred acquisition device and a preferred method for acquiring and/or determining a sleep state and/or for determining a sleep state of a vehicle occupant, including movements and heart rates thereof, reference is made in particular to publication EP 2712300 A1, the content of which is hereby incorporated by reference.

Regarding a preferred acquisition device and a preferred method for acquiring and/or determining a sleep state of a vehicle occupant, reference is made in particular to publication DE 10 2008 038022a1, the content of which is hereby incorporated into the present application. A wake-up device is proposed therein to be able to achieve the best possible sleep and to wake up gently a person sleeping in particular in a motor vehicle. The wake-up device should be equipped with a sensor device, which recognizes the movements of the sleeping person and thus preferably recognizes the so-called awake phase. Furthermore, the wake-up device should be equipped with a lighting device, which in the active state promotes continuous illumination of the surroundings of the sleeping person. Furthermore, the wake-up device should be equipped with a time measurement device for determining a wake-up time or a wake-up period, wherein the sensor device is configured such that it activates the lighting device at the wake-up time or during the wake-up period when a wake-up phase of a sleeping person is identified.

With regard to a preferred acquisition device and a preferred method for acquiring and/or determining a fatigue state and/or a sleep state of a vehicle occupant, reference is made in particular to publication DE 10 2017 220 120.7, the contents of which are hereby incorporated into the present application. A neurofeedback system is described which should preferably simplify relaxation in the cabin until falling asleep, and in which fatigue and/or sleep states can be detected.

The acquisition device is preferably adapted and determines (in particular for determining and/or acquiring a sleep state and/or a sleep stage), in particular from a preset set of sleep stages and/or a sleep stage of the occupant. The preset set of sleep stages is preferably stored in a storage device. The predefined sleep stage groups preferably have at least two different sleep stages, preferably at least three and particularly preferably at least five and in particular just five sleep stages.

The sleep state of the occupant may be present here if one (each) of the preset sleep stages of the occupant or the preset sleep stages are present or can be acquired and/or can be determined.

The state of consciousness, in particular the sleep state, sleep stage is preferably acquired and/or determined and/or identified by brain wave measurements and/or EEG (electroencephalogram, recording of voltage fluctuations at the surface of the head), in particular by deriving summed cortical potential fluctuations, and preferably by evaluation.

It is preferably determined whether the occupant is in a sleep state, and in particular a so-called non-REM sleep period (periods I-III) or in a REM sleep state. The different sleep stages are preferably distinguished from each other by means of an EEG.

In particular, healthy sleep includes five sleep stages. Sleep stages or sleep periods may be identified at brain activity. Here, the person's sleep remains substantially the same from falling asleep until deep sleep. This situation changes suddenly with the onset of so-called REM sleep or dream sleep. REM stage is named with the rapid eye movement ("rapid eye movement") characteristic to this sleep segment. At this stage, brain activity accelerates significantly, and EEG shows many small spikes. This is similar to the sleep stage. Dream occurs especially in the REM stage. Sleep researchers hypothesize that during REM stage, emotional sensory impressions are primarily processed, but information is also processed.

Preferably, five sleep stages, (especially in sleep state), period I (light sleep), (sleep) period II, (sleep) period III (transition to deep sleep), (sleep) period IV (deep sleep) and REM sleep. The acquisition device preferably determines whether one of these sleep periods and which of these sleep periods is present in the occupant.

The (sleep) period I describes the sleep-in phase, i.e. the phase shortly after falling asleep. Preferably, during sleep period I, the brain switches from alpha waves to theta waves (4 to 7 Hz) (in EEG). Muscle tone decreases and conscious perception of the environment slowly disappears. Here, this may be during and after falling asleep, or the first few minutes shortly after falling asleep. At this stage, the organisms sink more and more calm and relax. Here, the pulse slows down, the respiration deepens, and the number of breaths decreases. This sleep period I is often experienced as a time of becoming heavy or falling, which is perceived as resembling a dream. Here, the sleep is still very shallow, and even a small disturbance will wake the sleeper, so that it is immediately awake,

during sleep period II, the θ wave continues to appear (in EEG), especially so-called sleep spindles and K-complexes are involved in this stage. During the sleep period of eight hours, the sleep period becomes longer and accounts for more than 50% of the total sleep.

In sleep period III (transition to deep sleep), delta waves (0.1 to <4Hz, slow waves with high amplitude) occur (in EEG) and are particularly prominent (20-50% of brain waves measured). Muscle tone continues to decline.

In sleep period IV (deep sleep), delta waves occur, which now constitute more than 50% of the measured brain waves. This is the deepest sleep stage. A sleeper who wakes up during this sleep stage may become disoriented and drowsy accordingly. During this sleep stage, sleep disturbances such as dreaminess, speaking during sleep, may occur.

REM sleep (also known as dream sleep or abnormal sleep) can be distinguished from other sleep stages in many respects. EEG is similar to sleep period I in that the dominant theta wave occurs. During REM sleep, rapid, unidirectional movement of the eye ball at a frequency of 1 to 4Hz is produced at regular intervals. The dream reporting of wakefulness in this phase is significantly more vivid, intuitive and emotional than wakefulness in other phases. Skeletal muscle is relaxed to the greatest extent during REM sleep, whereas eye muscle is absent. Resulting in activation of most vegetative functions with increased blood pressure, respiratory frequency and heart rate and increased blood flow to the genitalia.

In the case of a healthy person with good sleep, the formation of sleep stages is repeated a plurality of times per night. A complete cycle lasts approximately one half hour (plus or minus 10 minutes). From which about 50 minutes is apportioned to the fall-or light-sleep stage. The deep sleep and REM phases constitute about 40 minutes of this sleep cycle. At the beginning of the night, deep sleep predominates, and at the end of the sleep time, the sleeper stays more in the REM stage and prepares for waking up.

The so-called light sleep stage constitutes about half of sleep. It is also referred to as sleep period 2. Sometimes sleep periods 1 and 2 merge into a light sleep stage. At this stage, brain activity is primarily limited to low frequencies. Consciousness is turned off and the sleeper relaxes muscle sleep with little eye movement.

The deep sleep stage is divided in particular into sleep stages 3 and 4. This is the most deep stage of body relaxation. Many studies have shown that the deep sleep stage is the most valuable stage for physical and mental recovery. At this stage, many people have difficulty waking up. Thus, the person who has just awakened can only slowly retrieve consciousness.

In the alert state of a person, beta waves may be measured or acquired in an EEG. The alert state of the occupant may preferably be identified by measuring the beta wave (by means of EEG).

In a relaxed state with eye closure (or awake state) an alpha wave (8 to 13 Hz) will occur. The awake state of the occupant is preferably identified by measurement of the alpha wave (by means of EEG).

It is preferably evaluated whether it is an input caused by a sleep state and/or sleep disturbance, depending on which sleep stage and/or which sleep period is acquired and/or present. An evaluation is made as to whether it is an input caused by a sleep state and/or sleep disturbance, depending on whether deep sleep stages and/or shallow sleep stages are acquired and/or present. It is preferably evaluated whether it is an input caused by sleep states and/or sleep disturbances, depending on whether REM sleep and/or non-REM sleep is present.

In a further preferred method, the at least one vehicle user profile is stored in a memory device of the communication system. In this case, it is deduced whether a command relating to the vehicle function is to be implemented and/or whether the vehicle function can be operated and/or activated, depending on the detected and/or determined state of consciousness, in particular sleep, of the occupant and/or depending on a predefined vehicle function. The above occupant-specific features and/or indications are preferably stored in a vehicle user profile.

Preferably, it is possible to store in the vehicle user profile how certain vehicle functions are switched to be inoperable and/or not activatable depending on the identified sleep stage and fatigue state.

In a further preferred method, the blocking of the implementation of the instruction is released only after an additional confirmation of the instruction by the occupant, as a function of the acquired and/or ascertained state of consciousness of the occupant, in particular the sleep state. This provides the advantage that, although the security of the intentional input is ensured by blocking, the possibility for operation at any point in time is still provided.

The blocking may be a temporary blocking. Preferably, the blocking may be released after confirmation by the occupant or another occupant.

In a further preferred method, the health state of the occupant (and preferably also of all other occupants) is monitored, in particular as a function of the state of consciousness, in particular the state of sleep, of the occupant. The health status of the occupant is preferably monitored during the sleep time of the occupant.

In the case of a vehicle occupant, it should preferably not only be possible to carry out sleep states or even sleep stage detection or acquisition of sleep stages, but also to monitor the vehicle occupant as far as possible with respect to its state of health during the sleep time.

For a preferred monitoring of the health status of the occupants, reference is made in particular to publication "K27561" [ please replace ], the content of which is hereby incorporated into the present application. The recognition of a healthy emergency, described therein, is preferably used, if any, of the occupants of the vehicle in the cabin. The advantageous behavior can thus be implemented such that the vehicle user is at as little risk as possible and/or the emergency program is activated preferably after the identification of a health emergency. In this way, for example, biological vital data of the vehicle occupants can be acquired and the health status of the vehicle occupants can be monitored, in particular permanently, in the passenger compartment and, if possible, an emergency program can be activated.

In a further preferred method, the assistance system supports the occupant when precautions are taken against a source of danger present in the vehicle interior space, which is associated with neurological disorders, in particular sleep disorders, of the occupant. This provides the advantage that potentially dangerous sources can be eliminated.

If the occupant has previously (i.e. before using the vehicle and/or if it is likely to also intend to sleep there) already known that the occupant is affected by a sleep disturbance, such as a dream, supportive precautions can be implemented in the cabin to protect himself or other travel partners in the cabin from injury. It is well known that the risk of accidents for nocturnal visitors is high, especially in unfamiliar and unusual circumstances. The nocturnal agent should lock with its object injuring itself or others, so it remains inaccessible to the nocturnal agent.

In this way, on the one hand, potential nocturnals can be advantageously alerted and if possible actively supported by the auxiliary system to safely store objects from the cabin.

In a further preferred method, at least one object in the vehicle interior is detected by means of an image acquisition device, in particular an interior camera, which is a potential hazard source for the occupant in relation to neurological disorders, in particular sleep disorders.

The interior space camera may help (preferably by means of image processing and object recognition and preferably by means of algorithms such as artificial intelligence, machine learning and/or deep learning etc.) find potentially dangerous objects within the cabin, especially before sleeping time, and inform the vehicle user about it, especially. A recommendation may be made to the vehicle user or occupant as to how to safely store such objects (e.g., in a trunk or lockable storage space, which may only be opened if possible in an awake state). For example, advice for storing objects may be conveyed audibly through an animated display on a screen in the vehicle and/or through a broadcast.

Potentially dangerous vehicle functions during a dream trip (e.g., opening a window or door of an autonomous vehicle) may be set to be inoperable by the vehicle user or by the occupant himself (in the case of a dream trip).

Preferably, it is possible to agree with the vehicle user or occupant that the vehicle function is still released or activated if the vehicle user or occupant makes an agreement confirmation, such as naming a preset keyword or answering a (preset) question. This provides the advantage that it can be bypassed in case the vehicle side sleep recognition is not active. It is preferably ensured that the vehicle user or passenger in particular can identify the health emergency as well as possible at any time.

In a further preferred method, the communication system provides a notification signal and/or a warning signal for in particular an acoustic output as a function of the state of consciousness, in particular of the sleep state, in particular of a preset sleep stage (for example sleep stage IV) of the occupant and/or preferably in the event of a neurological disorder, in particular a (in particular preset) sleep disorder.

Such notification and/or warning methods may be activated, for example, if the occupant outputs with respect to the communication system as a (known) sleeper and/or a person suffering from another sleep disorder and/or the communication system recognizes this (per se, for example by comparison with a database and/or recognition of the occupant).

The notification signal and/or the warning signal can be guided and/or output to the passenger and/or to another passenger. In the case of known purchasers, when the first sign of initial dream can be identified in the case of potential purchasers, the traveler (e.g., spouse) in the same car can be alerted or awakened early. The hotel partner may take action early or sooth early or even wake up the nocturnal traveler. This behaviour is particularly interesting if a dream with violence is known in the case of a cyclist (see e.g. https:// www.spiegel.de/gesuphenit/psychrologie/seltsame-stonengen-wenn-schlabende-zu-schlaegin-werden-a-854389. Html).

The invention also relates to a communication system for a vehicle, in particular for taking into account a sleeping state of an occupant, comprising at least one input device for inputting by the vehicle occupant at least one instruction relating to at least one vehicle function of the vehicle, wherein the communication system is adapted and determined to cause the implementation of the instruction relating to the vehicle function by providing an implementation signal in response to the input.

According to the invention, the communication system has an acquisition device for acquiring and/or determining a conscious state, in particular a sleep state, of the occupant upon the input of the command, and the communication system is adapted and determines whether an evaluation is made on the basis of the conscious state, in particular the sleep state, which is caused by the conscious state, in particular the sleep state, and/or by a neurological disorder, in particular a sleep disorder.

It is therefore also proposed in the context of the communication system according to the invention to evaluate whether the input made by the occupant is a conscious and/or intentional input as a function of the acquired or defined state of consciousness, in particular sleep state, and preferably sleep stage. This provides the advantage that countermeasures, in particular for preventing implementation, can be introduced before an unintentional input by the occupant is implemented.

The communication system is preferably set up, adapted and/or determined to carry out the above-described method and to carry out all the method steps already described above in connection with the method, alone or in combination with each other.

The communication system is preferably adapted and determines that the state of consciousness, in particular the sleep state, of the occupant is taken into account when the instruction is implemented, in particular by modifying and/or preventing and/or delaying it.

The invention also relates to a vehicle, in particular a motor vehicle, comprising a communication system for a vehicle as described above. The vehicle interior space may be equipped with at least one acquisition device for acquiring a state of consciousness of the occupant, in particular a sleep state, and/or at least one and preferably a plurality of input devices. The vehicle may in particular be a (motor) road vehicle. In principle, the proposed method and the proposed communication system can be used for all automation levels of a vehicle.

The vehicle may be a motor vehicle, in particular a motor vehicle controlled by the driver himself ("driver only"), a semi-automatic, automatic (e.g. automatic level 3 or 4 or 5 (SAE J3016 standard)), or an autonomous motor vehicle. Where automatic level 5 refers to fully automatic driving of the vehicle. The vehicle may also be an unmanned transport system. Here, the vehicle may be controlled by the driver or autonomously driven.

The invention also relates to a computer program or a computer program product comprising program means, in particular program code, which represent or encode at least the method steps of the method according to the invention and which are preferably one of the described preferred embodiments and are designed for implementation by a processor device.

The invention also relates to a data storage on which at least one embodiment of the computer program according to the invention or a preferred embodiment of the computer program is stored.

Drawings

Further advantages and embodiments emerge from the drawing:

wherein:

fig. 1 shows a schematic illustration of a vehicle with a communication system according to the invention according to one embodiment.

Detailed Description

Fig. 1 shows a schematic diagram of a vehicle with a communication system according to the invention according to one embodiment. Reference numeral 20 here schematically denotes a communication system for the vehicle 1.

Reference numeral 10 denotes an occupant of the vehicle 1. In the embodiment shown in fig. 1, the occupant is in a lying position (lieeposition) and is in a sleep state.

Reference numeral 22 designates an input device of the communication system 20. Here, it may be, for example, a mobile terminal device, but it may also be a physical actuating element. The occupant may make inputs to the communication system via the input device 22 and thereby communicate and trigger instructions regarding the vehicle function, such as opening a window.

Sleep disturbances, such as dreaming and speaking during sleep, may occur if the occupant is in, inter alia, a deep sleep stage, in particular stage IV. If this is the case in the case of an occupant, there is a risk that the occupant, during a section of the dream, for example, takes an input at the input device 22 (unintentionally) and thus (unintentionally) causes an implementation of the input by the occupant at the input device 22 by the communication system 20.

Reference numeral 24 designates an acquisition device, such as a camera, which can determine and/or acquire the sleep state of the occupant, and preferably also the sleep stages. Based on the acquired and/or determined sleep states, the communication system may evaluate or evaluate inputs made by the occupant as inputs caused by sleep disturbances (e.g., by dream). Based on this evaluation, the implementation of the input instruction may be caused, modified, delayed and/or prevented. If the communication system, for example, generates an evaluation, the communication system can prevent the execution of the input command and preferably additionally prevent further inputs by the occupant 10.

Applicant reserves all features disclosed in the claims to be essential to the invention as long as they are novel relative to the prior art, alone or in combination. It is furthermore pointed out that the following figures also describe features which may be advantageous per se. Those skilled in the art will immediately recognize that certain features depicted in the drawings may also be advantageous without employing other features in the drawings. Furthermore, those skilled in the art will recognize that advantages may also result from a combination of features shown in the various figures or in different figures.

REFERENCE SIGNS LIST

1. Vehicle with a vehicle body having a vehicle body support

10. Occupant and method for producing the same

20. Communication system

22. Input device

24. Acquisition apparatus

Claims

1. Method for operating a communication system (20) for a vehicle (1), comprising at least one input device (22) for inputting at least one command for at least one vehicle function of the vehicle (1) by an occupant (10) of the vehicle (1), wherein the communication system (20) is adapted and determined to cause the execution of the command by providing an execution signal in response to the input, characterized in that a state of consciousness, in particular a sleep state, of the occupant (10) is acquired and/or determined upon the input of the command, and the communication system (20) is evaluated on the basis of the state of consciousness, in particular the sleep state, whether it is an input caused by the state of consciousness, in particular the sleep state, and/or a neurological disorder, in particular a sleep disorder.

2. Method according to claim 1, characterized in that the sleep state is taken into account when implementing the instructions, in particular by means of modification and/or blocking and/or delaying.

3. Method according to any of the preceding claims, characterized in that at least one vehicle user profile is stored in a memory device of the communication system, from which it is deduced whether instructions concerning the vehicle function are implemented and/or whether the vehicle function can be operated and/or activated from the acquired and/or determined sleep state of the occupant and/or from a preset vehicle function.

4. Method according to any of the preceding claims, characterized in that, depending on the acquired and/or determined sleep state of the occupant (10), the blocking of the execution of the instruction is released only after an additional confirmation of the instruction by the occupant. .

5. Method according to any of the preceding claims, wherein the health status of the occupant is monitored, in particular in accordance with the sleep status of the occupant.

6. The method according to any one of the preceding claims, wherein an assistance system supports the occupant when implementing preventive measures for a source of risk of sleep disturbance of the occupant present in the vehicle interior.

7. Method according to any of the preceding claims, characterized in that objects in the vehicle interior are identified by means of an image acquisition device, in particular an interior space camera, which is a potential hazard source with respect to sleep disturbance of the occupant.

8. Method according to any one of the preceding claims, characterized in that the communication system provides a notification signal and/or a warning signal for, in particular, an acoustic output depending on the sleeping state of the occupant (10).

9. Communication system (20) for a vehicle (1), comprising at least one input device (22) for inputting at least one instruction concerning at least one vehicle function of the vehicle (1) by an occupant (10) of the vehicle (1), wherein the communication system (20) is adapted and determines that the implementation of the instruction concerning the vehicle function is caused in response to the input by providing an implementation signal, characterized in that the communication system has an acquisition device (24) for acquiring and/or determining a state of consciousness, in particular a sleep state, of the occupant upon the input of the instruction, and the communication system (20) is adapted and determines to evaluate, based on the state of consciousness, in particular the sleep state, whether it is an input caused by the state of consciousness, in particular the sleep state and/or a neurological disorder, in particular a sleep disorder.

10. Vehicle (1), in particular motor vehicle, comprising a communication system (20) for a vehicle according to the preceding claim.