DE102019210928A1 - Monitoring device for a vehicle, method and computer program - Google Patents

Abstract

The invention relates to a monitoring device (2) for an interior of a vehicle (1). The invention also relates to a method which is designed in particular to be carried out on the monitoring device according to the invention, a driver assistance system and a computer program.

Description

The invention relates to a monitoring device for an interior of a vehicle, having a first sensor, which is designed to detect the interior using first interior data in a first physical quantity, and wherein the first sensor is designed to forward the first interior data to an analysis unit, and
at least one second sensor, which is designed to detect the interior space based on second interior space data in a second physical variable different from the first, and wherein the at least second sensor is designed to forward the at least second interior space data to the analysis unit. The invention also relates to a method which is designed in particular to be carried out on the monitoring device according to the invention, a driver assistance system and a computer program.

Humans have the ability to perceive their surroundings and to react appropriately to this perception. The driver can be supported while driving by functions such as ABS (anti-lock braking system), ESP (Electronic Stability Control) and automated emergency systems.

However, modern vehicles are increasingly being operated by driver assistance systems in an autonomous or automated mode of operation and can be designed entirely as driverless means of transport. The interaction of microprocessor systems, sensors and actuators enables the vehicle to drive largely independently.

The autonomous mode of operation of a vehicle can be divided into different levels of autonomy. These are subdivided into a first level of autonomy, which is designed with driver assistance, via, for example, a fourth and fifth level of autonomy, in which the control of the vehicle is taken over permanently and fully automatically by the system and a driver is no longer required. With such full automation, no human intervention is required other than setting the goal and starting the system.

Such highly automated systems are particularly advantageous for taxis and buses or other public transport.

However, the automated driving style means that dangerous situations to which a reaction must be made are no longer recorded by the driver. This is the case even if a driver is present but, due to the almost automatic driving style, he does not have to concentrate on the vehicle occupants or the surroundings. This can lead to dangerous situations in the vehicle.

• einem globalen Positionierungssystem, das dem Datenprozessor elektrische Signale bereitstellt, wobei die Daten einen Ort eines Fahrzeugs darstellen;
• einem Notfallsensor, der geeignet ist, einen Fahrzeugnotfall zu erkennen und dem Datenprozessor elektrische Signale bereitzustellen, die die Notsituation anzeigen; einem Sprachgenerator, der konfiguriert ist, elektrische Signale, die den Ort des Fahrzeugs und eine Notrufbenachrichtigung darstellen, in Sprachsignale umzuwandeln; einen Lautsprecher zum hörbaren Weitergeben einer Notrufbenachrichtigung an den Fahrzeuginsassen;
• eine Eingabevorrichtung, die geeignet ist, einen Abbruch des Notrufs durch den Insassen als Reaktion auf die Notrufbenachrichtigung zu empfangen; und ein Kommunikationsmodul, das angeordnet ist, um eine erste Kommunikationsverbindung mit einem ersten Mobiltelefon innerhalb des Fahrzeugs herzustellen;
• einen Verbindungsstatus mit dem ersten Mobiltelefon auf den Verlust der ersten drahtlosen Kommunikationsverbindung zu überwachen;
• eine Anzeige für den Fahrzeuginsassen zu erzeugen, wenn die erste drahtlose Kommunikationsverbindung verloren gegangen ist;
• und automatisch eine zweite drahtlose Kommunikationsverbindung mit einem zweiten Mobiltelefon innerhalb des Fahrzeugs herzustellen; wenn ein Notruf nicht vom Insassen abgebrochen wird, Initiieren eines Anrufs an einen Ersthelfer und Übermitteln
• von Sprachsignalen, die den Ort des Fahrzeugs darstellen, an das Mobiltelefon für eine Telefonkommunikation mit einem Ersthelfer.

The EP 2170657 B1 discloses, as a monitoring unit, a device suitable for informing a first responder of a vehicle emergency, the device comprising a data processor in communication with the following:

• a global positioning system that provides electrical signals to the data processor, the data representing a location of a vehicle;
• an emergency sensor which is suitable to detect a vehicle emergency and to provide the data processor with electrical signals indicating the emergency situation; a voice generator configured to convert electrical signals representative of the location of the vehicle and an emergency notification to voice signals; a loudspeaker for audibly relaying an emergency notification to the vehicle occupant;
• an input device adapted to receive an abandonment of the emergency call by the occupant in response to the emergency call notification; and a communication module arranged to establish a first communication link with a first mobile phone within the vehicle;
• monitor a connection status with the first mobile phone for loss of the first wireless communication link;
• generate an indication to the vehicle occupant when the first wireless communication link is lost;
• and automatically establish a second wireless communication link with a second mobile phone within the vehicle; if an emergency call is not canceled by the occupant, initiate a call to a first responder and transmit
• of voice signals representing the location of the vehicle to the mobile phone for telephone communication with a first responder.

It is therefore an object of the invention to provide means with which the safety for vehicle occupants can be increased.

This object is achieved by a monitoring device with the features of claim 1, a driver assistance system with the features of claim 13, and a method with the features of claim 14.

The object is also achieved by a computer program with the features of claim 15.

The subclaims list further advantageous measures that can be combined with one another in any suitable manner in order to achieve further advantages.

The object is achieved by a monitoring device for an interior of a vehicle, having a first sensor, which is designed to detect the interior based on first interior data in a first physical quantity, and wherein the first sensor is designed to forward the first interior data to an analysis unit is
and at least one second sensor, which is designed to detect the interior space on the basis of second interior space data in a second physical variable different from the first, and wherein the at least second sensor is designed to forward the at least second interior space data to the analysis unit,
in which
the analysis unit is included, which is designed to merge the first interior space data with the at least second interior space data, and which is also designed to identify an emergency situation based on the merged interior space data, and which is also designed to generate a warning based on an identified emergency situation .

A classification of sensors can take place, for example, on the basis of physical quantities. Physical quantities can be, for example, pressure, weight, acceleration, light intensity, temperature, radiation, sound, magnetic flux, speed and many other physical quantities. For example, a sound transducer, for example a microphone for recording sound, or an image sensor, for example a light sensor / optoelectronic sensor (camera) for recording light, can be provided for the measurement. Such light sensors / optoelectronic sensors are, for example, photocells or image sensors such as CCD (charge-coupled device) sensors, Super-CCD sensors, or CMOS sensors (active pixel sensors). Furthermore, temperature sensors are provided in vehicles, for example, which supply an electrical signal as a measure of the temperature.

In this context, the term sensor can also be understood to mean only that interface which receives and forwards the corresponding sensor data.

A fusion is a combination of the first interior space data recorded by the first sensor with the second interior space data recorded by the at least second sensor. There can also be several sensors for generating several interior data, which are merged with one another. After the merger, the merged data are processed further by the analysis unit. For a suitable fusion of the sensor data, different algorithms adapted depending on the sensor type can be used.

Emergency situations can be abnormalities / dangers or an abnormal behavior. Furthermore, an emergency situation can also affect a single vehicle occupant, such as a heart attack, vomiting, fainting, etc.

An analysis unit can for example be a data processing unit which has one or more processors and / or graphics processors, for example as SOC (system on chip). Furthermore, the analysis unit can, however, also be designed as a software module or a combination of software and hardware modules.

The analysis unit can include a method from the field of machine learning to identify the emergency situation on the basis of the merged interior data. For this purpose, the analysis unit can have a trained artificial neural network, for example. Furthermore, the identification of the emergency situation can also be achieved through a target / actual comparison with predefined situations.

An emergency situation can be recognized particularly reliably by using at least two sensors that deliver different measured variables and / or are mounted at different points on the vehicle, as well as the subsequent fusion of the sensor data and identification by an analysis unit based on the merged sensor data. In this way, incorrect assessments can be largely avoided.

Such a monitoring device according to the invention can ensure public order and the safety of the vehicle occupants in vehicles, in particular in autonomous or automated vehicles.

For example, in the case of driving services, ride sharing or car sharing, unexpected situations can arise both as a result of events external to the vehicle and internal events which can be identified by means of the monitoring device according to the invention.

For example, if a professional driver is present in the vehicle, for example a taxi or a bus, many situations in the rear area of the vehicle that are not visible to the driver are recorded, identified and, in the event of an emergency situation identified, sent to a central point forwarded. In this way, dangerous situations for the professional driver or other vehicle occupants can be avoided, or help can be requested in good time.

The monitoring device according to the invention achieves a higher level of security for the vehicle occupants. The monitoring device according to the invention is particularly useful, for example, both for autonomous driving services and for car sharing.

If the sensors are only designed as an interface, the monitoring device according to the invention can be designed as a software module.

The analysis unit is preferably designed to classify the identified emergency situation and to generate a warning based on the classified emergency situation. In this way, a warning adapted to the emergency situation can be generated. A situation-specific warning is thus generated on the basis of the classified emergency situation.

The first sensor is preferably designed as an optical sensor, in particular as an image sensor. This is preferably designed as a front camera or front camera system for recording the interior. A plurality of optical sensors are advantageously provided for detection, which are arranged at different positions in the vehicle interior. With the help of optical sensor data, vehicle occupants and situations can be recognized. Gesture recognition, for example, can be recognized by the optical sensor. Using the gesture recognition, a danger for the vehicle occupant (s), e.g. physical violence, can be recognized. Furthermore, the development of smoke, for example due to a fire due to a technical defect, or smoking while driving, can be detected optically. In addition, face recognition / person identification can be carried out using the optical interior data, and the number of people can be identified and compared with the invoice, for example when taking a taxi / bus ride.

Furthermore, active or passive safety systems, for example seat belt systems, can be recognized and possibly activated depending on the person's position, for example in connection with seat and position recognition sensors.

Furthermore, the second sensor is preferably designed as a sound sensor. This converts an acoustic signal into electrical signals or vice versa. Examples of sound transducers are microphones or loudspeakers. By training as a sound transducer, acoustic abnormalities such as loud cries for help can be perceived. The acoustic sensor and, if necessary, speech recognition, can easily identify dangers such as annoyance or threats to the vehicle occupants.

Through a combination of optical interior data and acoustic interior data, emergency situations can be easily identified by the analysis unit and preferably then classified. For example, in the event of a violent situation, it can be better recognized optically, for example by gestures, and acoustically, for example by shouts for help, and the urgency / selection of a warning can be better adapted to the situation.

A bidirectional communication module is preferably provided, the monitoring device being designed to transmit the warning to a monitoring center via the bidirectional communication module. The corresponding warning is only transmitted to the monitoring center when an emergency situation has been identified, that is to say only when necessary, for example a detection of an emergency situation such as abnormalities / dangers. This ensures data protection.

In a preferred embodiment, the warning includes the transmission of the merged interior data to the monitoring center. The monitoring center can provide help in the event of danger. The presence of a monitoring device which is connected to a monitoring center means that dangers can be avoided in advance, since the vehicle occupants adapt their behavior due to a transmission in the event of danger or danger to other vehicle occupants. In the event of administrative offenses, people can also be temporarily or permanently blocked by the monitoring center, for example for transport by bus / train / taxi.

Such a monitoring center represents a higher level of security for the vehicle occupants. The monitoring center can access the interior data, for example image / video data, as well as audio data and, if necessary, store them. Data protection is guaranteed through transmission and storage to the monitoring center only in an emergency.

The bidirectional communication module is preferably designed to receive image and / or sound data sent by the monitoring center, the monitoring device being designed to output the image and / or sound data. Thus, the monitoring center can also visually and acoustically immediately with the output of image and sound data (voice data) Intervene in an emergency. The image data can be presented, for example, on a display unit, for example a head-up display or infotainment display. The sound data are output through loudspeakers, for example.

The bidirectional communication module is preferably set up to receive vehicle parameters sent by the monitoring center and to forward the received vehicle parameters to a driver assistance system in order to adapt the current vehicle parameters. In this way, for example, in the event of an acute dangerous situation, for example a fire in the vehicle interior, driving can be stopped immediately. This serves for the further safety of the vehicle occupants.

The bidirectional communication module is preferably designed as a wireless radio link. Wireless radio links can in particular be established via a cellular telephone network, preferably via GSM (Global System for Mobile communications) or streaming or edge technology.

The warning preferably includes the transmission of an emergency call to a corresponding emergency service point. This can be, for example, the fire brigade, emergency doctor, a center of a mobility service provider etc. in order to get help as quickly as possible in an acute dangerous situation. Additionally or alternatively, an emergency call button can also be provided for this purpose, so that the vehicle driver, passenger or passenger can intervene depending on the situation and make an emergency call.

A further sensor is preferably provided, the further sensor being designed as a temperature sensor for detecting the interior temperature of the interior of the vehicle as further interior data, and the temperature sensor being designed to forward the further interior data to the analysis unit. In this way, for example, a dangerous rise in temperature can be detected in summer, which is a danger above all for older people or children. The temperature sensor data are merged by the analysis unit with the first and at least second interior data for further processing and for the identification and subsequent classification of an emergency situation. If, for example, a warning with a dangerous temperature is transmitted to the monitoring center, it can generate a set of vehicle climate control parameters and / or settings for the vehicle interior and transmit it, for example, via the bidirectional communication module. The communication module preferably forwards this to the driver assistance system for control.

The analysis unit preferably includes a method from the field of machine learning for identifying the emergency situation on the basis of the merged interior data. Machine learning can include algorithms from the field of supervised or unsupervised learning. In this context, machine learning can be understood as the generation of knowledge from experience, in which learning algorithms develop a complex model from training data sets. The trained model can then be applied to unknown data of the same type. Machine learning can be understood as a data analysis method that enables the automatic creation of analysis models. Machine learning is a branch of artificial intelligence. Such algorithms are well suited for the merged interior data.

In a further preferred embodiment, the analysis unit for identifying the emergency situation on the basis of the merged interior space data comprises a trained artificial neural network.

In particular, this can be designed as a deep neural network in order to carry out or apply deep learning. Deep learning performed or applied by an artificial neural network can employ one or more algorithms.

The artificial neural network comprises input nodes, one or more hidden layers and output nodes. For classifications involving images, pixel values of an input image that forms part of the classification can be assigned to input nodes and then fed through the network. At the end of the calculation, the output node can yield a value that corresponds to a class that was derived from the neural network.

In order for a neural network to differentiate between classes, it must be taught on the basis of training data sets.

The trained artificial neural network is preferably designed as a convolutional neural network (CNN) or as generative adversarial networks (GANs). These are particularly suitable in connection with optical and acoustic sensors and optical and acoustic sensor data.

In a further preferred embodiment, a further sensor is provided, the further sensor being designed to detect the vehicle environment using vehicle environment data and to forward the vehicle environment data to the analysis unit, and the analysis unit to merge the first interior data with the at least second interior data and the vehicle environment data is designed, and which is also designed to identify an emergency situation based on the merged interior space data and vehicle environment data.
This means that events in the vehicle environment can be combined with events in the vehicle interior in order to identify an emergency situation. Examples of the application can be a robbery started from outside the vehicle, a critical dynamic driving situation (e.g. the vehicle breaking out of the lane), or an unintentional departure from the planned route (e.g. due to a wet / icy road surface Leave the lane / street).

The object is also achieved by a driver assistance system with a monitoring device as described above, the driver assistance system being designed to control the vehicle in an autonomous or automated mode of operation. Such a monitoring device can increase the driving safety of the vehicle occupants in an autonomous or automated vehicle. Dangerous / emergency situations are reliably recognized and a warning is issued. This warning can preferably be displayed in the vehicle interior itself and also sent to the monitoring center. The monitoring center can react with appropriate actions, for example informing the police, emergency doctor, headquarters of a mobility service provider and generating appropriate vehicle parameters for setting the corresponding vehicle components by the driver assistance system, for example for locking the vehicle, or rapid changes to the planned trajectory.

• - Erzeugen von ersten Innenraumdaten des Innenraums durch einen ersten Sensor, wobei der erste Sensor zur Erfassung einer ersten physikalischen Größe ausgebildet ist,
• - Weiterleiten der ersten Innenraumdaten an eine Analyseeinheit durch den ersten Sensor,
• - Erzeugen von zweiten Innenraumdaten des Innenraums durch einen zweiten Sensor, wobei der zweite Sensor zur Erfassung einer zweiten, von der ersten unterschiedlichen physikalischen Größe ausgebildet ist,
• - Weiterleiten der zweiten Innenraumdaten an die Analyseeinheit durch den zweiten Sensor,
• - Fusionieren der ersten Innenraumdaten mit den zumindest zweiten Innenraumdaten durch die Analyseeinheit,
• - Identifizieren einer Notsituation anhand der fusionierten Innenraumdaten durch die Analyseeinheit,
• - Erzeugen einer Warnung bei einer identifizierten Notsituation durch die Analyseeinheit.

Furthermore, the object is achieved by a method for monitoring an interior of a vehicle by means of a monitoring device as described above with the steps:

• - Generating first interior data of the interior by a first sensor, the first sensor being designed to detect a first physical variable,
• - Forwarding of the first interior data to an analysis unit by the first sensor,
• - Generating second interior data of the interior by a second sensor, the second sensor being designed to detect a second physical variable different from the first,
• - Forwarding of the second interior data to the analysis unit by the second sensor,
• - Fusion of the first interior data with the at least second interior data by the analysis unit,
• - Identification of an emergency situation using the merged interior data by the analysis unit,
• - Generating a warning in the event of an identified emergency situation by the analysis unit.

An emergency situation can be recognized particularly reliably by using at least two sensors which supply different measured variables, as well as the subsequent fusion of the sensor data and identification by an analysis unit on the basis of the merged sensor data. In this way, incorrect assessments can be largely avoided.
The method can thus be designed as a software module.

Furthermore, the object is achieved by a computer program comprising commands which cause the monitoring device as described above to carry out the method as described above. The computer program can be stored on a portable memory, for example a memory stick or a CD-ROM, and installed, for example, when visiting a workshop. The computer program can be designed as a data carrier signal in order to enable simple transmission via cloud / external computer to a suitable vehicle.

• 1: ein Fahrzeug mit einem erfindungsgemäßen Fahrerassistenzsystem und einer erfindungsgemäßen Überwachungseinrichtung in einer ersten Ausgestaltung,
• 2: ein erfindungsgemäßes Verfahren,
• 3: ein Fahrzeug mit einem erfindungsgemäßen Fahrerassistenzsystem und einer erfindungsgemäßen Überwachungseinrichtung in einer zweiten Ausgestaltung.

Further features of the present invention emerge from the following description with reference to the accompanying figures. They show schematically:

• 1 : a vehicle with a driver assistance system according to the invention and a monitoring device according to the invention in a first embodiment,
• 2 : a method according to the invention,
• 3 : a vehicle with a driver assistance system according to the invention and a monitoring device according to the invention in a second embodiment.

Although the invention has been described in more detail by means of the preferred exemplary embodiments, the invention is not restricted by the examples disclosed.

1 shows a vehicle 1 with a monitoring device according to the invention 2 , which is integrated here in a driver assistance system.

The monitoring device according to the invention 2 has several as a first sensor Cameras 3 for optical detection of the vehicle interior. These are arranged in the vehicle interior to detect the front area of the vehicle interior, the rear area of the vehicle interior and the central area of the vehicle interior. The cameras 3 image sensors are preferred, and different optical sensors can also be used. The use of multiple cameras 3 at different positions in the vehicle interior also guarantees optical detection in not so accessible / clear places in the vehicle interior.
The monitoring device according to the invention 2 has several sound transducers as a second sensor, here as microphones 4th designed on. Using multiple microphones 4th increases the intelligibility of speech, for example if someone has suffered a heart attack and can only speak in a soft and incomprehensible manner. The microphones 4th are preferably arranged at different points in the vehicle interior.

Other sensors are speakers 5 arranged in the vehicle interior, which is an output of a monitoring center 10 enable transmitted sound / sound data. Alternatively, the microphones 4th also be designed to output the audio data.

A smoke detector can also be used 6 be arranged in the vehicle interior. This can be designed to detect fire or cigarette smoke, etc. An emergency button can also be used 7th be provided, which for example with an analysis unit 11 as well as directly with the monitoring center 10 connected is.

There is also a temperature sensor 12th provided for determining a temperature in the vehicle interior.

Furthermore is the monitoring center 10 intended. A controller can be provided in this for monitoring and evaluating the interior data sent in the event of an emergency. However, it is also conceivable for such an evaluation to be carried out with the aid of a computer.

The cameras 3 who have favourited microphones 4th and the speaker 5 , the smoke alarm 6 , the emergency button 7th as well as the temperature sensor 12th are with the analysis unit 11 in the vehicle 1 connected.

Furthermore, a belt sensor, seat occupancy sensor and other sensors, not shown, can also be present and with the analysis unit 11 be connected. Furthermore is the analysis unit 11 intended to merge the interior data and identify an emergency situation. The analysis unit 11 preferably comprises a data processing unit with, for example, a plurality of microprocessors and graphics processors, memory units, etc. The analysis unit also comprises 11 a trained artificial neural network to identify and preferably also classify the emergency situation on the basis of the merged interior data. Furthermore, the emergency situation can also be identified, for example, by a target / actual comparison with predefined situations.

The trained artificial neural network is preferably designed as a convolutional neural network (CNN) or as generative adversarial networks (GANs). Convolutional Neural Networks extract features from input images and unfold these images using filters. Convolutional neural networks have several layers of convolution. The structure of a classic convolutional neural network consists of one or more convolutional layers, followed by a pooling layer. The convolutional layer is the plane of the fold. This layer can be applied as often as you like. If the repetition is sufficient, it is referred to as a deep convolutional neural network. The pooling layer condenses and reduces the resolution of the recognized features in the convolutional layer.

A convolutional neural network works robustly and is insensitive to distortion or other optical changes. The Convolutional Neural Network can process images recorded under different lighting conditions and in different perspectives. Nevertheless, it recognizes the typical features of an image and is therefore particularly suitable for image and audio recognition.

Generative Adversarial Networks (GANs) is a method based on unsupervised learning. Generative Adversarial Networks consist of two artificial neural networks, the generator and the discriminator. GANs are also used for image and audio recognition.

Other AI (Artificial Intelligence) processes can also be used.

The interior data recorded by the various sensors are sent to the analysis unit 11 transmitted, which recognizes an emergency situation by means of the trained artificial network and if an emergency situation is recognized, a warning, for example the interior data, to the monitoring center 10 transmits.

Since only when necessary, that is, when an emergency situation is detected, for example abnormalities / dangers, such transmission and, if necessary, storage of the data by the Monitoring center 10 data protection is guaranteed. The warning includes the transmission of the interior data, i.e. audio and sound data, temperature, smoke development, etc.

There is also a display unit 8th provided, which is designed for example as a head-up display or as a monitor, and a representation of the monitoring center 10 transmitted video / image data in the vehicle interior.

A bidirectional radio interface is also used as the communication module 9 provided for transmitting indoor data to the monitoring center 10 , as well as to receive the from the monitoring center 10 transmitted sound and image data for output on the display unit 8th and the speaker 5 . The transmitted audio and video data can be sent to the driver assistance system or to the monitoring device 2 be forwarded, which then manages the output.

Through the monitoring center 10 In order to clarify and to increase the safety of the vehicle occupants, an image and sound transmission can first be transmitted from the controller in the monitoring center 10 to the vehicle 1 be accomplished. The image and sound data are transmitted by the radio interface 9 received and by the monitoring device 2 or the driver assistance system on the corresponding devices, here the display unit 8th and the speaker 5 issued. This means that there is direct audio and video communication from the controller of the surveillance center 10 possible with the vehicle occupants. This increases the safety level for the vehicle occupants.

Furthermore, from the monitoring center 10 in certain emergency situations vehicle parameters are generated, which are generated by the radio interface 9 can be received and used by the driver assistance system to control various vehicle components. For example, if smoke is generated by fire, the vehicle can be brought to a standstill and the door can be opened.

As a result, problematic vehicle occupants can, for example, be prevented from continuing their journey in autonomously driving buses / taxis.

The monitoring center can also 10 issue a warning if a discrepancy is found between the number of vehicle occupants transported and the number of vehicle occupants paying. Such a warning could be, for example: Beware of fraud.

In the non-autonomous vehicle, the vehicle driver, who, for example, through the monitoring center 10 has been warned to intervene in an emergency, for example harassment, and make an emergency call. In the autonomous vehicles, the emergency call is made, for example, by the monitoring device 2 discontinued.

By such a monitoring device 2 both public safety and public order can be ensured.

2 shows a method according to the invention in a first embodiment.

Here the vehicle points 1 ( 1 ) the monitoring device 2 ( 1 ) on those with cameras 3 ( 1 ), Microphones 4th ( 1 ) and the smoke detector 6 ( 1 ) and the temperature sensor 12th ( 1 ) in a first step S1 Records interior data.

In a second step S2 the interior data are sent to the analysis unit 11 ( 1 ) and evaluated by it. If no emergency situation is detected, the data are recorded up to the end of the journey and continuously analyzed during the journey until the end of the journey.

In a third step S3 becomes an emergency situation by the analysis unit 11 ( 1 ) recognized.

In a fourth step S4 the interior data are transmitted through the radio interface 9 ( 1 ) to the monitoring center 10 ( 1 ) transmitted. In addition, a first classification can also be transmitted, for example: “Warning, fire” if through the microphones 4th ( 1 ), for example screams "fire", and the temperature sensor 12th ( 1 ), the smoke detector 6 ( 1 ) and the cameras 3 ( 1 ), corresponding interior data have been generated and identified and classified with regard to this emergency situation.

In a fifth step S5 Depending on the situation, communication between the vehicle occupants and the monitoring center is established 10 ( 1 ), for example the controller. For this purpose, image data are sent via the radio interface 9 ( 1 ) for representation on the display unit 8th ( 1 ) and audio data via the radio interface 9 ( 1 ) for output through the loudspeaker 5 ( 1 ) from the monitoring center 10 ( 1 ) transmitted.

If necessary, it can be used in parallel or subsequently in one sixth step S6 from the monitoring center 10 ( 1 ) an emergency call can be made, for example to a fire brigade, police, emergency doctor etc.

In a seventh step S7 can, depending on the situation, parallel to the fifth step S5 or sixth step S6 from the monitoring center 10 ( 1 ) Vehicle parameters are generated, which via the radio interface 9 ( 1 ) can be sent to the driver assistance system. The driver assistance system can now control the vehicle with these vehicle parameters 1 ( 1 ) Taxes. This is the case, for example, if there is an outbreak of fire or harassment. In this case, for example, vehicle parameters can be generated and transmitted which enable the vehicle to be stopped immediately 1 ( 1 ) and ensure that the vehicle doors are opened. 3 shows a vehicle 1 with a driver assistance system according to the invention and a monitoring device 2a according to the invention in a second embodiment.

The monitoring device 2a also has cameras as sensors 3 , Microphones 4th , Speaker 5 and the smoke alarm 6 as well as the temperature sensor 12th as well as the display unit 8th and the radio interface 9 on. The monitoring device 2a also has the analysis unit connected to the sensors 11 on.

In addition, the monitoring device 2a has radar sensors 13 and / or lidar sensors 14th and / or image sensors 15th which on the vehicle 1 for recording the surroundings of the vehicle 1 are arranged. The radar sensors 13 and / or lidar sensors 14th and / or image sensors 15th generate vehicle environment data, which the environment of the vehicle 1 map in terms of data.

These vehicle environment data are now sent to the analysis unit 11 ( 1 ) Posted. This now identifies an emergency situation based on the interior data sent and the vehicle surroundings data. For example, an inevitable accident, if the vehicle occupants indicate it using interior data such as facial and sound expressions, can be identified in advance in connection with the vehicle environment data. This also makes it clear to the inspector why, for example, the vehicle occupants are reacting in panic. This also makes it possible to make an emergency call in advance, for example to the police or the emergency doctor.

The lidar sensors 14th , Radar sensors 13 and image sensors 15th can also enable the vehicle to operate autonomously 1 be used.

The monitoring device 2 , 2a is particularly suitable for autonomous vehicles 1 which have no driver and, for example, no steering wheel or accelerator pedal. Such vehicles 1 are currently preferably designed as minibuses, and mostly have an electric motor that can accelerate up to 45 km / h to 130 km / h. By the monitoring device according to the invention 2 , 2a the so-called operators, who accompany all journeys and can intervene if necessary, can be dispensed with. As a result, such minibuses can be used over a large area without additional costs for the operator in each of the minibuses. Use in so-called robo-shuttles is also possible, or in conventional vehicles.

By the monitoring device according to the invention 2 , 2a acoustic abnormalities, such as loud cries for help, can be perceived and identified as an emergency situation. Furthermore, through the optical image recognition and a speech recognition by means of the analysis unit 11 Hazards such as annoyance and a threat to vehicle occupants are recognized. By means of the sensor data of the at least acoustic and at least optical sensor, the fusion of the sensor data and the analysis unit 11 In connection with this, gestures such as physical violence can be recognized. Furthermore, smoke development, for example due to fire, for example due to a technical defect or smoking while driving, can be detected and then by means of the monitoring center 10 responded.

Furthermore, face recognition by the analysis unit 11 be made.

The number of people can also be identified and compared with the billing. Active or passive safety systems can be through the monitoring device 2 , 2a can be activated depending on the position of the person.

An emergency situation can be reliably identified by using at least two sensors which measure different variables. Acoustic sensors can record a cry for help and the optical sensors record a threatening gesture, for example. Furthermore, a rising temperature with a tired facial expression and beads of sweat can indicate a dangerous interior temperature as an emergency situation.

By connecting to a monitoring center as required 10 a higher level of safety is created for the vehicle occupants.

Furthermore, through the vehicle intervention by means of the monitoring center 10 transmitted vehicle parameters can be responded to in a targeted manner in emergency situations.

List of reference symbols

1

vehicle

2

Monitoring device

3

camera

4th

microphone

5

speaker

6th

smoke detector

7th

Emergency button

8th

Display unit

9

Radio interface

10

Monitoring center

11

Analysis unit

12

Temperature sensor

13

Radar sensor

14th

Lidar sensor

15th

Image sensor

S1, .., S7

Procedural steps

QUOTES INCLUDED IN THE DESCRIPTION

This list of the documents listed by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Patent literature cited

• EP 2170657 B1 [0007]

Claims (15)

Monitoring device (2) for an interior of a vehicle (1), having a first sensor which is designed to detect the interior using first interior data in a first physical quantity, and wherein the first sensor is also used to forward the first interior data to an analysis unit ( 11), and at least one second sensor, which is designed to detect the interior space on the basis of second interior space data in a second physical variable different from the first, and wherein the at least second sensor for forwarding the at least second interior space data to the analysis unit ( 11), characterized in that the analysis unit (11) is comprised, which is designed to merge the first interior data with the at least second interior data, and which is also designed to identify an emergency situation based on the merged interior data, and which is designed for this purpose is designed based on e to generate a warning in an identified emergency situation. Monitoring device (2) Claim 1 , characterized in that the analysis unit (11) is designed to classify the identified emergency situation and to generate a warning based on the classified emergency situation. Monitoring device (2) Claim 1 or 2 , characterized in that the first sensor is designed as an optical sensor and the second sensor as a sound sensor. Monitoring device (2) according to one of the preceding claims, characterized in that a bidirectional communication module is provided, and the monitoring device (2) is designed to transmit the warning to a monitoring center (10) via the bidirectional communication module. Monitoring device (2) Claim 4 , characterized in that the warning comprises transmitting the merged interior data to the monitoring center (10). Monitoring device (2) Claim 4 or 5 , characterized in that the bidirectional communication module is designed to receive image and / or sound data sent by the monitoring center (10), and the monitoring device (2) is designed to output the image and / or sound data. Monitoring device (2) according to one of the preceding Claims 4 to 6 , characterized in that the bidirectional communication module is set up to receive vehicle parameters sent by the monitoring center (10) and to forward the received vehicle parameters to a driver assistance system for adapting the current vehicle parameters. Monitoring device (2) according to one of the preceding claims, characterized in that the warning includes transmitting an emergency call to a corresponding emergency service point. Monitoring device (2) according to one of the preceding claims, characterized in that a further sensor is provided, wherein the further sensor is designed as a temperature sensor for detecting the interior temperature of the interior of the vehicle (1) as further interior data and wherein the temperature sensor for forwarding the further Interior data to the analysis unit (11) is formed. Monitoring device (2) according to one of the preceding claims, characterized in that the analysis unit (11) for identifying the emergency situation based on the merged interior space data comprises a method from the field of machine learning. Monitoring device (2) Claim 10 , characterized in that the analysis unit (11) comprises a trained artificial neural network to identify the emergency situation on the basis of the merged interior data. Monitoring device (2) according to one of the preceding claims, characterized in that a further sensor is provided, wherein the further sensor is designed to detect the vehicle environment by means of vehicle environment data and to forward the vehicle environment data to the analysis unit (11) and where the analysis unit (11) is designed to merge the first interior space data with the at least second interior space data and the vehicle environment data, and which is also designed to identify an emergency situation based on the merged interior space data and vehicle environment data. Driver assistance system with a monitoring device (2) according to one of the preceding claims, characterized in that the driver assistance system for controlling the vehicle (1) is designed in an autonomous or automated mode of operation. Method for monitoring an interior of a vehicle (1) by means of a monitoring device (2) according to one of the previous Claims 1 to 12th , characterized by the steps: - generating first interior data of the interior by a first sensor, the first sensor being designed to detect a first physical variable, - forwarding the first interior data to an analysis unit (11) by the first sensor, - generating second interior data of the interior by a second sensor, the second sensor being designed to detect a second, different physical variable from the first, - forwarding of the second interior data to the analysis unit (11) by the second sensor, - merging of the first interior data with the at least second interior data by the analysis unit (11), - identification of an emergency situation on the basis of the merged interior data by the analysis unit (11), - generation of a warning in the event of an identified emergency situation by the analysis unit (11). Computer program, comprising instructions which cause the monitoring device (2) according to one of the preceding Claims 1 to 12th the procedure after Claim 14 executes.

Images