LU100894B1

LU100894B1 - Method for Robust Vehicle Occupancy Detection with Vital Sign Monitoring

Info

Publication number: LU100894B1
Application number: LU100894A
Authority: LU
Inventors: Oscar Gomez; Muhammad-Zeeshan Khan; Jochen Landwehr; Dimitri Tatarinov; Peter Larsen
Original assignee: Iee Sa
Priority date: 2018-08-03
Filing date: 2018-08-03
Publication date: 2020-02-04

Abstract

A radar sensor system (10) and a method of operating the radar sensor system (10) for detecting an occupancy in an interior (34) of a vehicle (32) and with vital sign monitoring is provided. The radar sensor system (10) comprises a radar transmitting unit (12), a radar receiving unit (18) and a signal processing and control unit (26). The method includes at least steps of transmitting (60) a radar wave towards a scene within the vehicle interior (34), receiving (62) at least one radar wave that has been generated by reflection of the transmitted radar wave (16), - decomposing (64) the received at least one radar wave (22) into range, Doppler and angular information, quantifying and tracking (68) a movement in each region of interest (38, 40, 42) by angular gating and range gating, detecting (78) and monitoring vital signs of occupants (28, 30) in each region of interest (38, 40, 42), and determining (84) whether quantified and tracked movements in the scene are related to an occupant (28, 30) or to external or internal disturbances, based on a fulfillment of at least one predefined condition concerning the quantified and tracked movements and/or the detected and monitored vital signs.

Description

Method for Robust Vehicle Occupancy Detection with Vital Sign Monitoring Technical field

[0001] The invention relates to a method of operating a radar sensor system for detecting an occupancy in an interior of a vehicle with vital sign monitoring, to a radar sensor system configured for automatically executing such method and a software module enabling automatically executing the method.

Background of the Invention

[0002] It is known in the art to use radar technology to detect objects of interest within an occupancy zone. For instance, US 2010/0225764 A1 describes a detection system and a method for such occupancy detection. The detection system can be applied to monitor vehicles or objects that could be used to transport unwanted materials into an area. The presence of objects of interest can include pedestrians, bicycles, automobile bumpers, glass surfaces, and vehicle wheel frames. The detection system comprises a plurality of sensors, which may be formed as frequency-modulated continuous-wave multi-beam radar sensors, including an entry sensor having a field of view oriented to detect object entry with respect to an entry portion of an occupancy volume, and an exit sensor having a field of view oriented to detect object exit with respect to an exit portion of the occupancy volume, the exit portion being different than the entry portion. The detection system further includes at least one approach sensor having a field of view oriented to detect object approach relative to the entry portion of the occupancy volume, and at least one retreat sensor having a field of view oriented to detect object retreat relative to the exit portion of the occupancy volume.

[0003] It is also known in the art to use radar technology for automotive seat occupant detection systems. Occupancy sensors based on radar technology offer advantages in comparison to other occupancy detection methods as their operation is contact-free and unnoticeable for vehicle occupants. Moreover, radar sensors can easily be integrated in the vehicle interior, for example behind plastic, plastic covers and diverse materials such as textiles, wood, stone, concrete, glass, and so forth.

[0004] Vehicle seat occupancy detection systems are nowadays widely used in vehicles, in particular in passenger cars, for instance for detection of left-behind pets and/or children, vital sign monitoring, vehicle seat occupancy detection for seat belt reminder (SBR) systems, or anti-theft alarm. Such vehicle seat occupancy detection systems can be employed for providing a seat occupancy signal for various appliances, for instance for the purpose of a seat belt reminder (SBR) system or an activation control for an auxiliary restraint system (ARS).

[0005] An output signal of a seat occupant detection and/or classification system is usually transferred to an electronic control unit of the vehicle to serve, for instance, as a means of assessing a potential activation of an installed vehicle passenger restraint system.

[0006] Further valuable information, usable as important input for Advanced Driver Assistance Systems (ADAS) could be provided by monitoring a vital sign of the detected person.

[0007] In US 7,036,390 B2, a method for detecting a human body in a vehicle is described. A synthetic wave is obtained, which represents the synthesis (i.e. the superposition) of a transmitted wave radiated from a sensor and a reflected wave returned from a breathing human body, and the presence or absence of a human in the vehicle is detected from the envelope of the synthetic wave. The frequency of the transmitted wave is 4.7 GHz or higher. When the presence of a human is detected continuously for a predetermined length of time, it is determined that a human is present in the vehicle. When the interior of the vehicle reaches a temperature dangerous to a human, the sensor is activated. A device for monitoring temperature gradient inside the vehicle is provided, and the sensor is activated in advance when it is predicted by the device that the interior of the vehicle is likely to reach a dangerous temperature. When the rate of change of temperature inside the vehicle is mild, the sensor is operated at random timing for a predetermined length of time. When the interior of the vehicle reaches a dangerous temperature, and when the presence of a human is detected, an alarm is issued.

[0008] Furthermore, from US 6,753,780 B2 a motion sensing system and method is known for detecting an occupant in a vehicle with enhanced sensitivity to detect small movement, such as movement caused by heartbeat and breathing of anoccupant. The system includes a radar motion sensor located in a compartment of the vehicle. The radar sensor transmits and receives signals within the compartment and generates sensed signals. The system has a controller for converting the sensed signals to a frequency domain. The controller further processes the frequency domain of sensed signals and determines the presence of movement of an occupant due to one of heartbeat and breathing of the occupant. To this end, the controller compares the frequency domain of sensed signals within the frequency range to a predetermined frequency characteristic, and further identifies a frequency of the sensed signals indicative of movement of an occupant due to one of heartbeat and breathing.

[0009] WO 2015/140333 A1 describes a method for ascertaining whether an unattended child is present within an automotive vehicle. The method uses a radar sensor system comprising a transmitter, and at least one sensor and processing circuitry, and exploits a breathing motion detected by radar signals, for instance by applying autocorrelation and peak finding. The method comprises - illuminating at least one occupiable position within the vehicle with radiation, the radiation exhibiting multiple frequencies, - generating radar sensor signals from radiation reflected as a result of the transmitted radiation, a plurality of the radar sensor signals corresponding to different frequencies; - operating the processing circuitry for generating, based on the radar sensor signals, a first indicator value, for instance R-value, the first indicator value indicating a degree of motion associated with the occupiable position; - determining whether the first indicator value satisfies a first predetermined criterion; - if the first indicator value satisfies the first predetermined criterion, generating, based on radar sensor signals, a second indicator value, the second indicator value indicating a degree of repetitive pattern within the radar sensor signals, for instance breathing pattern; and - determining that an unattended child is present within the automotive vehicle if the second indicator value satisfies a second predetermined criterion.

[0010] The second indicator value may comprise a breathing signature representing the extent to which the sensor signals indicate that motion indicative of infant breathing child is detected.

[0011] Further, WO 2015/022358 A1 describes a method for sensing an occupancy status within an automotive vehicle using a radar sensor system. The radar sensor system comprises an antenna system, at least one sensor and a processing circuitry. The method comprises illuminating, using the antenna system, at least one occupiable position within the vehicle with continuous wave (CW signals), wherein the CW signals can be frequency modulated in time. At least one sensor signal reflected as a result of the CW signals is received using at least one sensor, wherein the at least one sensor is defining a plurality of receive channels (1, ...,i). Each receive channel has a different frequency (fi, ..., fi). Processing circuitry is operable for applying, for each receive channel (1, ...,i), DC offset removal to the corresponding sensor signal to generate a modified signal; and generating, based on the modified signals, one or more occupancy status signals. In case of an employed FMCW radar sensor system, FFT processing is used to obtain the range of a detected occupant, and the vital signs of the occupant are obtained from the phase of the range FFT. The occupancy status signal indicate a property related to the at least one occupiable position. A system for carrying out the method is also disclosed. The techniques provide for in-vehicle occupant detection and classification (airbag suppression), for passenger presence detection and passenger's vital sign monitoring and for seat-belt reminder functionality (SBR). The radar system may consist of one or two (or four) outputs (/Q) per channel. The received signal may be mixed down with a single channel demodulator or an I/Q demodulator into the baseband. An advanced signal processing including an auto calibration routine and clutter removal may be used to detect occupants and their vital sign (breathing and heart beat) and to determine their distance to the radar system.

[0012] In the article “Non-Contact Estimation at 60 GHz for Human Vital Signs Monitoring Using a Robust Optimization Algorithm” by Ting Zhang et al, Conference IEEE APS 2016, Jun 2016, Fajardo (Porto-Rico), United States, 2016, AP-S/URSI 2016. <hal-01340613>, an approach to estimate body movements related to vital activities by means of a 60 GHz Doppler radar is described, usingrobust optimization algorithms including signal autocorrelation analysis in order to extract heart-rate and breathing information from the radar signals.

[0013] This document shall hereby be incorporated by reference in its entirety with effect for those jurisdictions permitting incorporation by reference.

[0014] It is therefore known in the art that a presence of a single vehicle passenger can be detected by conducting electromagnetic measurements such as Doppler radar techniques to measure, for instance, the passenger’s breathing or heartbeat. Unfortunately, received radar waves are corrupted with noise if, for instance, a vehicle is moving over a rough surface, in the presence of strong wind gusts, in the presence of engine vibrations or in a carwash. This noise, which could be of high amplitude compared to the signal of a passenger that is desired to detect will inevitably lead to an increased number of false alarms, i.e. falls positives or false negatives. Object of the invention

[0015] It is therefore an object of the invention to provide a method of robust vehicle occupancy detection with vital sign monitoring using radar technology, which is capable of reliably and robustly distinguishing between signals related to an occupant positioned in a specific region of interest in the vehicle interior and disturbing signals caused by vehicle interior events and/or events outside the vehicle, such as vehicle or sensor shaking induced by rough road, strong wind gusts, engine vibrations or car wash. General Description of the Invention

[0016] In one aspect of the present invention, the object is achieved by a method of operating a radar sensor system for detecting an occupancy in an interior of a vehicle. The radar system comprises a radar transmitting unit that is configured for transmitting radar waves towards a scene within the vehicle interior, and a radar receiving unit that is configured for receiving radar waves that have been transmitted by the radar transmitting unit and have been reflected by an object within the scene.

[0017] Further, the radar sensor system includes a signal processing and control unit that is at least configured to derive range information, Doppler information andangle-of-arrival information from the received radar waves. Methods for deriving these types of information are well known to those skilled in the art and therefore need not to be described in more detail herein. Any method that appears suitable to those skilled in the art may be applied.

[0018] The phrase “being configured to”, as used in this application, shall in particular be understood as being specifically programmed, laid out, furnished or arranged. Further, in this application the terms “radar wave” and “radar signal” are used as synonyms, wherein the term “radar wave” may be used to more specifically describe an electromagnetic wave of a radar frequency traveling in free air, and the term “radar signal’ shall be understood to relate either to an electromagnetic wave of a radar frequency that is carried by a transmission line or to a wire-bound signal that has been obtained by processing a received radar wave, for instance an IF (intermediate frequency) signal obtained by mixing a received radar wave with a local oscillator signal and/or demodulation.

[0019] The method includes at least the following steps, which are to be carried out in an iterative manner: - operating the radar transmitting unit for transmitting a radar wave towards a scene within the vehicle interior, - operating the radar receiving unit for receiving at least one radar wave that has been generated by reflection of the transmitted radar wave, - operating the signal processing and control unit for o decomposing the received at least one radar wave into range, Doppler and angular information, o quantifying and tracking a movement in each region of interest of a plurality of predefined different regions of interest by angular gating and range gating, o detecting and monitoring vital signs of occupants in each region of interest, and o for each region of interest, determining whether quantified and tracked movements in the scene are related to an occupant or to external or internal disturbances, based on a fulfillment of at least one predefined condition concerning the quantified and tracked movements and/or the detected and monitored vital signs.

[0020] The phrase “vital signs of an occupant’, as used in this application, shall in particular be understood to encompass at least one out of heartbeat and breathing of the occupant. The term “occupant”, as used in this application, shall in particular be understood to encompass human beings and pets.

[0021] It is an insight of the invention that by using a combination of range, Doppler and angle-of-arrival information, a robust vehicle occupancy detection method can be provided. The method enables to detect, quantify and track different movements in different regions of the scene. Internal and external motion can then be detected, separated and rejected or suppressed, for instance by range gating. The disclosed method is thus capable of reliably and robustly distinguishing between signals related to an occupant positioned in a specific region of interest in the vehicle interior and disturbance signals caused by vehicle interior events and/or events outside the vehicle.

[0022] Further, a presence of several occupants in the vehicle interior can be detected and their positions with respect to the radar sensor system be determined, and respective vital signs can be monitored simultaneously. Strong signals or disturbances that are present only outside a region of interest, e.g. a region including a seat row, do not affect any vital sign signals of occupants. As a consequence, a number of false alarms can be reduced and a robustness of the radar sensor system can be increased.

[0023] A non-limiting example for the at least one predefined condition could be that if strong movement is present in all or several range regions of interest, a strong disturbance is likely to be present. In contrast, if movement is detected only in a region with a seating position, and vital signs are found, it is quite certain to assume that the seat is occupied.

[0024] In preferred embodiments of the method, the step of determining whether quantified and tracked movements in the scene are related to an occupant or to external or internal disturbances is based on a temporal development of fulfillments of the at least one predefined condition concerning the quantified and tracked movements and the detected and monitored vital signs. In this way, a robustness of the method can be increased.

[0025] Preferably, the at least one predefined condition concerning the quantified and tracked movements and the detected and monitored vital signs includes consideration of a constraint regarding a number of passengers in the vehicle interior to be constant while the vehicle is driving. By use of this consideration, a plurality of potential occupancies in the interior of the vehicle can be ruled out, and a faster reactivity of the method can be achieved without compromising a sensitivity to false alarms.

[0026] Preferably, the step of decomposing the received radar waves into range, Doppler and angular information includes applying a fast Fourier transform to an intermediate frequency (IF) signal of the received radar waves for converting the IF signal of the received radar waves from the time domain into the range domain.

In suitable embodiments, the range information can readily be obtained in this way.

[0027] In preferred embodiments of the method, the step of decomposing the received radar waves further includes removing a static scene from the fast Fourier transform of the IF signal of the received radar waves by making use of a fast Fourier transform of an IF signal of at least one previously received radar wave. By removing the static scene, movements in the balance of the fast Fourier transform of the IF signals of the received radar waves can be detected more easily.

[0028] The static scene may be removed by, without being limited to, - subtracting the fast Fourier transform of IF signals of the received radar waves computed in the previous iteration from the fast Fourier transform of IF signals of the currently received radar waves, or - subtracting an average of a plurality of fast Fourier transforms of IF signals of received radar waves, computed at previous iterations, or by - applying any other static scene removal method that appears to be suitable to those skilled in the art, such as applying a high pass filter or the like.

[0029] In preferred embodiments of the method, the step of quantifying and tracking a movement in each region of interest includes comparing an amount representing the quantified movement with at least one predefined detectionthreshold value for the quantity. This can allow for a fast and simple determination of a location of a maximum quantity within the gating.

[0030] Preferably, the at least one predefined detection threshold value for the quantity is a function of at least one out of vehicle speed and vehicle acceleration. In this way, the at least one predefined detection threshold value can be adjusted to match different driving situations and, by that, can reduce a number of false alarms. For instance, the at least one predefined detection threshold value may increase with increasing vehicle speed, at which larger disturbing signals by vehicle shaking induced by rough road driving are to be expected.

[0031] In preferred embodiments, determining a number of occupants in the vehicle interior can be supported by providing a signal from a wheel pressure sensor or sensors, or by providing analysis data from a wheel pressure sensor device. The wheel pressure information can be combined with information about the vehicle door activities for further support of determining the number of occupants.

[0032] In preferred embodiments of the method, the step of detecting and monitoring vital signs of occupants comprises calculating a phase signal from the fast Fourier transform of IF signals of the received radar waves at determined movements in each region of interest, and processing each phase signal by reconstructing the frequency domain signal into a phase-coherent time domain signal. In this way, the vital signs can readily be obtained from the received radar waves.

[0033] Preferably, the step of quantifying and tracking a movement in each region of interest includes calculating power integrals in predefined regions for each region of interest. In this way, a robust and reliable tracking of a movement in a specific region of interest can readily be accomplished.

[0034] In preferred embodiments of the method, the plurality of predefined different regions of interest includes at least one out of a range-azimuth, range- elevation or azimuth-elevation region of interest. In this way, a robustness of the method can be increased even more. In suitable embodiments of the radar sensor system, occupancy detection can be enabled for car interiors with larger cabin volume, which may, for instance, include several rows of seats. Further, detectionof several occupants at different range-angle regions of interest can be enabled, and simultaneous monitoring of vital signs of these occupants can be accomplished.

[0035] Preferably, the method comprises a step of operating the signal processing and control unit for generating an output signal that is indicative of a determined occupancy of the vehicle interior. In this way, information concerning the determined occupancy of the vehicle interior can be transferred and assessed in a fast manner, and appropriate measures, such as warnings by an advanced driver assistance system (ADAS) can quickly be taken if applicable.

[0036] In another aspect of the invention, a radar sensor system for detecting an occupancy in an interior of the vehicle is provided. The radar sensor system comprises a radar transmitting unit that is configured for transmitting radar waves towards a scene within the vehicle interior, and a radar receiving unit that is configured for receiving radar waves that have been transmitted by the radar transmitting unit and have been reflected by an object within the scene.

[0037] Further, the radar sensor system includes a signal processing and control unit that is at least configured to derive range information, Doppler information and angle-of-arrival information from the received radar waves, to control operation of the radar transmitting unit and the radar receiving unit and to automatically execute steps of the method disclosed herein.

[0038] The benefits described in context with the disclosed method of operating a radar sensor system for detecting an occupancy in an interior of a vehicle apply to the radar sensor system to the full extent.

[0039] In preferred embodiments of the radar sensor system, the radar transmitting unit includes a plurality of radar transmitting antennas and the radar receiving unit includes a plurality of radar receiving antennas, and the radar transmitting antennas and the radar receiving antennas are configured to operate in a multiple-input and multiple-output (MIMO) configuration. In this way, an angular resolution of the radar sensor system can be improved. Finer angular resolution facilitates better distinguishing ability between signals related to an occupant positioned in a specific region of interest in the vehicle interior anddisturbing signals caused by vehicle interior events and/or events outside the vehicle.

[0040] In one contemplated MIMO configuration with a plurality of transmitting antennas, each transmitting antenna is understood to be able to transmit radar waves in an independent manner that represent mutually orthogonal codes. Each receiving antenna is further understood to be able to receive radar waves that have been transmitted by any of the transmitting antennas and that have been reflected by an object without any cross talk disturbance.

[0041] Another contemplated MIMO configuration is the Time Division Multiplexing (TDM)-MIMO, which does not require orthogonal codes. Also, Binary Phase Modulated (BPM)-MIMO configurations are contemplated for use with this invention.

[0042] MIMO radar configurations provide benefits regarding enlarged size of virtual aperture, improved spatial resolution and less sensitivity to interference signals, as is well known in the art.

[0043] Preferably, the radar transmitting unit and the radar receiving unit form an integral unit and comprise a plurality of at least two transceiver antenna units, which are configured to work in a MIMO configuration. In this way, hardware parts can be saved and a more compact design of the radar sensor system can be achieved.

[0044] In preferred embodiments of the radar sensor system, the radar transmitting unit is configured to transmit frequency-modulated continuous radar waves. Frequency-modulated continuous wave (FMCW) radar sensor systems are well known in the art to be able to provide received radar waves from which range information, Doppler information and angle-of-arrival information can be derived.

[0045] In yet another aspect of the invention, a software module is provided for controlling automatic execution of the method disclosed herein.

[0046] The method steps to be conducted are converted into a program code of the software module, wherein the program code is implementable in a digital memory unit of the radar sensor system and is executable by a processor unit of the radar sensor system. Preferably, the digital memory unit and/or processor unit may be a digital memory unit and/or a processing unit of the signal processing andcontrol unit of the radar sensor system. The processor unit may, alternatively or supplementary, be another processor unit that is especially assigned to execute at least some of the method steps.

[0047] The software module can enable a robust and reliable execution of the method and can allow for a fast modification of method steps.

[0048] These and other aspects of the invention will be apparent from and elucidated with reference to the embodiments described hereinafter.

[0049] It shall be pointed out that the features and measures detailed individually in the preceding description can be combined with one another in any technically meaningful manner and show further embodiments of the invention. The description characterizes and specifies the invention in particular in connection with the figures.

Brief Description of the Drawings

[0050] Further details and advantages of the present invention will be apparent from the following detailed description of non-limiting embodiments with reference to the attached drawing, wherein: Fig. 1 schematically illustrates, in a side view, a configuration of an embodiment of a radar sensor system in accordance with the invention for detecting an occupancy in an interior of a vehicle and vital sign monitoring by executing an operating method in accordance with the invention, Figs. 2a and 2b show a flowchart of an embodiment of the method in accordance with the invention of operating the radar sensor system pursuant to Fig. 1, Fig. 3 shows a plot of range information after a step of decomposing received radar waves into range, Doppler and angular information, Fig. 4 is a detailed flowchart illustrating a step of quantifying and tracking a movement in regions of interest, Fig. 5 is a detailed flowchart illustrating an alternative step of quantifying and tracking a movement in regions of interest,

Fig. 6 schematically illustrates predefined different regions of interest covering a rear seat bench located in the vehicle passenger department pursuant to Fig. 1, Fig. 7 illustrates a step of determining seat occupancy based on a temporal development of fulfilments of a predefined condition including consideration of a constraint regarding the number of passengers, and Fig. 8 schematically illustrates fast and slow transitions between determined occupancies.

Description of Preferred Embodiments

[0051] Fig. 1 schematically illustrates, in a side view, a configuration of an embodiment of a radar sensor system 10 in accordance with the invention for detecting an occupancy in an interior 34 of a vehicle 32 with vital sign monitoring by executing an operating method in accordance with the invention. The interior 34 of the vehicle 32 is formed by a passenger car compartment of a sedan-type passenger car. Fig. 1 shows a side view of a passenger 28 occupying a driver's seat and another passenger 30 occupying one seat of three-seat rear bench 46 of the passenger car. More passengers (not shown) may be present, occupying a passenger front seat or other seats of the three-seat rear bench 46.

[0052] The radar sensor system 10 comprises a radar transmitting unit 12 that in this specific embodiment is configured to transmit frequency-modulated continuous radar waves (FMCW), and in this specific embodiment includes a plurality of three radar transmitting antennas 14. In other embodiments, the radar transmitting unit may be configured to transmit radar waves modulated by frequency-shift keying (FSK), or the radar transmitting unit may be configured to transmit radar pulses. Also, in other embodiments, the number of transmitting antennas may be different. In this specific embodiment, the radar transmitting unit 12 is configured for supplying the three radar transmitting antennas 14 with radar signals having a radar carrier frequency that is frequency-modulated with a saw-tooth pattern, as is known in the art, but other frequency modulation patterns such as triangular, sinusoidal modulation or any other pattern that appears suitable to those skilled in the art are also contemplated. In this specific embodiment, the radar carrier frequency of the radar sensor system 10 isselectable within a radar frequency range between 77 GHz and 81 GHz, but other frequency ranges, for instance a range between 57 GHz and 64 GHz, may be chosen, depending on the application. The radar transmitting antennas 14 are installed in a front region of the headliner and are directed rearwards. The radar transmitting unit 12 is configured for transmitting radar waves 16 via the radar transmitting antennas 14 towards a scene within the vehicle interior 34 given by the vehicle passenger compartment, and particularly towards a chest and an abdominal region of the driver 28 and other potentially present passengers 30.

[0053] The radar sensor system 10 further includes a radar receiving unit 18 having three radar receiving antennas 20 and being configured for receiving radar waves 22 that have been transmitted by the radar transmitting unit 12 and have been reflected by an object within the scene, among them the passengers 28, 30 that are present in the vehicle interior 34.

[0054] In this specific embodiment, each of the three radar transmitting antennas 14 is paired with one of the three radar receiving antennas 20 to be co- located in a monostatic arrangement, which is indicated in Fig. 1 by use of a combined symbol. In other embodiments, a bi-static arrangement or any other arrangement of an equal or unequal number of radar transmitting antennas and radar receiving antennas may also be possible. In this specific embodiment, the radar transmitting unit 12 and the radar receiving unit 18 each form an integral part of a transceiver unit 24, sharing common electronic circuitry and a common housing. In other embodiments, the radar transmitting unit and the radar receiving unit may be designed as separate units.

[0055] In this specific embodiment, the three transceiver antennas formed by the three radar transmitting antennas 14 and the three radar receiving antennas 20 are configured to operate in a multiple-input and multiple-output (MIMO) configuration.

[0056] In the MIMO configuration, each transceiver antenna is understood to be able to transmit radar waves in an independent manner that represent mutually orthogonal codes. Each transceiver antenna is further understood to be able to receive radar waves that have been transmitted by itself and any of the other transceiver antenna units and that have been reflected by an object, without any cross talk disturbance. The MIMO radar configuration provide benefits regardingenlarged size of virtual aperture, improved spatial resolution and less sensitivity to interference signals, as is well known in the art.

[0057] Furthermore, the radar sensor system 10 comprises a signal processing and control unit 26 that is connected to the radar transmitting unit 12 for controlling operation of the radar transmitting unit 12. The signal processing and control unit 26 is also connected to the radar receiving unit 18 for receiving radar signals generated by the radar receiving unit 18 from the received radar waves 22.

[0058] The signal processing and control unit 26 is configured to derive range information, Doppler information and angle-of-arrival information from the IF signals of the received radar signals. The signal processing and control unit 26 comprises a processor unit and a digital data memory unit (not shown) to which the processor unit has data access.

[0059] In the following, an embodiment of a method of operating the radar sensor system 10 for detecting an occupancy in the interior of the vehicle with vital sign monitoring will be described with reference to Fig. 1 and Figs. 2a and 2b in general, which provides a flowchart of the method. In preparation of operating the radar sensor system 10, it shall be understood that all involved units and devices are in an operational state and configured as illustrated in Fig. 1.

[0060] In order to be able to carry out the method automatically and in a controlled way, the signal processing and control unit 26 comprises a software module 36. The method steps to be conducted are converted into a program code of the software module 36. The program code is implemented in the digital data memory unit of the signal processing and control unit 26 and is executable by the processor unit of the signal processing and control unit 26.

[0061] Execution of the method may be initiated for instance, but not limited to, by turning on the passenger car ignition. Other initiation methods are also possible. As shown in Figs. 2a and 2b, the steps of the method are carried out in an iterative manner.

[0062] In a first step 60 of the method, the radar transmitting unit 12 is operated by the signal processing and control unit 26 for transmitting a radar wave 16 towards the vehicle passenger compartment. In another step 62, the radar receiving unit 18 is operated by the signal processing and control unit 26 forreceiving radar waves 22 that have been transmitted by the radar transmitting unit 12 and that have been generated by reflection at any object in the scene, and in particular at the passengers 28, 30 that are present in the vehicle passenger compartment, and, more specifically, by reflection at the chest and the abdominal region of the passengers 28, 30.

[0063] In a next step 64, the signal processing and control unit 26 is operated to decompose an IF signal of the received radar signal into range information, Doppler information and angular information. To this end, the signal processing and control unit 26 applies a fast Fourier transform (FFT) to the IF signal of the received radar signal, which is a time domain signal, for extracting the range information. By that, the IF signal of the received radar signal is converted to the frequency domain. In another step 66, a static scene is removed by subtracting an FFT of an IF signal of a previously received radar signal, for example an IF signal of a radar signal that has been received in the previous iteration.

[0064] Fig. 3 shows a plot of the range information after the step 64 of decomposing the IF signal of the received radar signal into range, Doppler and angular information and after the step 66 of removing the static scene. The range represented in Fig. 3 by range bins on the horizontal axis comprises a plurality of predefined regions of interest: a plurality of short-range regions of interest 38, a seat region of interest 40, which corresponds to a distance between the radar receiving unit 18 and the three-seat rear bench 46, and a plurality of long-range regions of interest 42. The range information of the FFT of the IF signal of the received radar signal shows a peak that is spread over a number of range bins in the seat region of interest 40.

[0065] In a next step 68 of the method, movement in each predefined region of interest 38, 40, 42 is quantified and tracked by angular gating and range gating.

[0066] To this end, a movement indicator ind, is calculated that represents a quantity of the movement. In this specific embodiment, the movement indicator indseat is given by a power integral to be calculated in predefined regions for each region of interest 38, 40, 42. Along the range information of the FFT of the IF signal of the received radar signal, maximum values are searched for by comparing the power integrals as amounts that represent the movement with apredefined detection threshold value for the power integral. If the amount exceeds the predefined detection threshold value, the range bin index of each one of the found maxima is placed into a buffer of predefined size. Then, the median of each buffer contents is calculated in every new iteration.

[0067] In this specific embodiment, the predefined detection threshold value for the power integral is a number that is constant for each predefined region of interest 38, 40, 42. In alternative embodiments, a predefined detection threshold value for the power integral may be a function of at least one out of vehicle speed and vehicle acceleration.

[0068] An example for detection of a disturbance is illustrated by Fig. 4, showing a detailed flowchart of the step 68 of quantifying and tracking a movement in three different regions of interest: a short-range region 38, the seat region 40 and a long-range region 42. For each region of interest 38, 40, 42 a detection threshold value thrçeat» thTsnort, throng for the power integral has been predefined. The various detection threshold values thrseat» thTspore, throng may be equal but do not have to be. In this specific embodiment, all three detection threshold values thrseat» tATshort thTiong are mutually different.

[0069] Whenever a movement indicator indseat» inds, ind, exceeds the detection threshold value three, thrspore throng Of the respective region of interest, a corresponding detection flag 44 is set to value “1”, with the meaning that significant movement has been detected in the respective region of interest 38, 40,

42. If significant movement has been detected in all three regions of interest 38, 40, 42, it is quite certain to assume that a strong disturbance is present, which may be caused for instance by shaking the vehicle 32 by strong wind gusts, or the like.

[0070] An alternative step of quantifying and tracking a movement in regions of interest is illustrated in the detailed flowchart of Fig. 5. Herein, the received radar signal FFT is divided in N short-range regions of interest, M long-range regions of interest and a seat region of interest.

[0071] The movement indicator inds;, i = 1,..,N of each short-range region is compared against a corresponding detection threshold Thr, as shown in Fig. 5. Whenever the indicator of the i** short-range region exceeds the correspondingdetection threshold, a detection flag ps; is set to value “1”, otherwise the variable detection flag p,; remains at value “0”. The values of ps;, à = 1, …, N are then used to compute a probability, which is compared to a probability threshold p,,, to determine if a short-range disturbance is present.

[0072] In a similar manner, movement indicators ind, ;, j = 1,...,M of each long- range region are compared against the corresponding detection threshold Thr, and long-range detection flags p,;,j = 1,…, M are obtained. A probability is calculated from values of the long-range detection flags p,;, which is compared to a probability threshold p,,, to determine if a long-range disturbance is present.

[0073] In a variation of this embodiment, variables pg;,i =1,..,N and p,;,j = 1, ..., M can take weighting values instead of “1” and “0”.

[0074] As described before, the three-seat rear bench 46 of the vehicle 32 lies in the range seat region 40 of interest pursuant to Fig. 3. A volume encompassing the three-seat rear bench 46 is further divided into three predefined regions of interest corresponding to a right seating position 48, a middle seating position 50 and a left seating position 52 (Fig. 6). The predefined regions of interest are defined in terms of range and azimuth (angular quantities), which can be converted to Cartesian coordinates x, y and z in the usual manner.

[0075] A movement indicator is calculated for each seating position 48, 50, 52 and compared to a detection threshold that corresponds to the respective seating position 48, 50, 52. To this end, a power integral is calculated in each predefined region, and a median of each buffer contents, which is the range bin index of each one of the found maxima, is calculated in every new iteration, as described before. Every new iteration of the method steps contributes to a temporal development of the buffers.

[0076] A buffer strategy regarding the temporal development of the buffer may be implemented similar to the one described in international application WO 2015/022358 A1, which shall hereby be incorporated by reference in its entirety with effect for those jurisdictions permitting incorporation by reference.

[0077] As another step 78 of the method, vital signs of occupants, formed by a breathing movement, are detected and monitored in each region of interest 38, 40,

42. This is accomplished in a step 80 by calculating a phase signal from the FFT of the IF signal of the received radar waves 22 at determined movements in each region of interest 38, 40, 42, i.e. at every median of the buffer contents, and processing each phase signal in a step 82 of reconstructing the frequency domain signal into a phase-coherent time domain signal.

[0078] For each region of interest 38, 40, 42, it is determined in another step 84 whether quantified and tracked movements in the scene are related to an occupant or to external or internal disturbances, based on a fulfillment of predefined conditions concerning the quantified and tracked movements and/or detected and monitored vital signs.

[0079] In this specific embodiment, a first predefined condition is that if movement is detected in a seat region of interest 40 only, and vital signs are detected, it is determined that the movements in the scene are related to a seat occupant occupying the specific seat region of interest 40. A second predefined condition is that if movement is detected in a short-range region of interest 38, a seat region of interest 40 and a long-range region of interest 42, it is determined that the movements in the scene are not related to a seat occupant but are related to the presence of a strong disturbance such as the vehicle 32 shaking by wind gusts or by rough road driving.

[0080] In another step 86 of the method, the signal processing and control unit 26 is operated for generating an output signal that is indicative of the determined occupancy of the vehicle interior 34.

[0081] In a variation of the method, the step 84 of determining whether quantified and tracked movements in the scene are related to an occupant or to external or internal disturbances is based on a temporal development of fulfilments of the predefined conditions concerning the quantified and tracked movements and the detected and monitored vital signs. In this variation of the method, the fulfiliment or non-fulfiliment of the predefined conditions has to be persistent for a predefined number of iterations, which relates to a predefined qualification time.

[0082] The qualification time can either be decreased to improve reactivity at an expense of a higher susceptibility to false alarms, or can be increased to improverobustness at an expense of a reduced reactivity, i.e. a detection of an occupant, who is changing seat positions, might take a longer time.

[0083] In order to overcome the long qualification time issue, the predefined conditions concerning the quantified and tracked movements and the detected and monitored vital signs includes consideration of a constraint regarding a number of passengers in the vehicle interior 34 to be constant while the vehicle 32 is operated with closed vehicle doors (Fig. 1).

[0084] Fig. 7 illustrates a step of determining seat occupancy based on a temporal development of fulfilments of a predefined condition including consideration of the above-mentioned constraint.

[0085] The occupancy states of the three-seat rear bench 46 can be represented by a 3-bit word, for instance (0,1,0), with the meaning: “Empty” (left seating position), “Occupied” (middle seating position), “Empty” (right seating position).

[0086] An occupant changing from one seating position to another is detected after a buffer value 54 that is related to a new seating position has been consecutively increased over a predefined time period 58 that is shorter than the qualification time, while another history buffer value 56 that is related to an old seating position has been consecutively decreased over the same time period 58. If this is detected, the occupancy state of the new seating position is set to “Occupied”, while the occupancy state of the old seating position is immediately set to “Empty”.

[0087] This occupancy state transition is carried out in any one of the cases of occupancy transitions with a constant number of occupants as shown in Fig. 8. These potential occupancy transitions, which can be called “fast transitions”, as they do not require any opening and closing of doors of the vehicle 32, are shown for the three-seat rear bench 46 of the vehicle 32 and one seat occupant on the left-hand side of Fig. 8. On the right-hand side of Fig. 8, potential “fast occupancy | transitions” for the three-seat rear bench 46 of the vehicle 32 and two seat occupants are shown. Potential occupancy transitions with a non-constant number of occupants, called “slow transitions”, require opening and closing of doors of the vehicle 32, and are therefore determined after the regular qualification time.

[0088] While the invention has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered illustrative or exemplary and not restrictive; the invention is not limited to the disclosed embodiments.

[0089] Other variations to be disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims. In the claims, the word “comprising” does not exclude other elements or steps, and the indefinite article “a” or “an” does not exclude a plurality, which is meant to express a quantity of at least two. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage. Any reference signs in the claims should not be construed as limiting scope.

List of Reference Symbols radar sensor system 50 middle seating position 12 radar transmitting unit 52 left seating position 14 radar transmitting antenna 54 buffer value 16 transmitted radar waves 56 buffer value 18 radar receiving unit 58 predefined time period radar receiving antenna indg; movement indicator short-range 22 received radar waves region of interest 24 transceiver unit ind; movement indicator long-range 26 signal processing and control region region of interest unit indseat movement indicator seat

28 passenger region of interest passenger psi short-range region detection flag 32 vehicle py; long-range region detection flag 34 vehicle interior per Probability threshold 36 software module Thr threshold value for movement 38 short-range region of interest indicator 40 seat region of interest thrsnort threshold value short-range 42 long-range region of interest region 44 flag three threshold value seat region 46 three-seat rear bench thTong threshold value long-range 48 right seating position regionee —————————————————

Claims

1. A method of operating a radar sensor system (10) for detecting an occupancy in an interior (34) of a vehicle (32), with vital sign monitoring; the radar sensor system (10) comprising - a radar transmitting unit (12) being configured for transmitting radar waves (16) towards a scene within the vehicle interior (34), - a radar receiving unit (18) being configured for receiving radar waves (22) that have been transmitted by the radar transmitting unit (12) and have been reflected by an object within the scene, and - a signal processing and control unit (26) that is at least configured to derive range information, Doppler information and angle-of-arrival information from the received radar waves (22); and the method including at least the following steps, which are to be carried out in an iterative manner: | - operating the radar transmitting unit (12) for transmitting (60) a radar wave towards a scene within the vehicle interior (34), - operating the radar receiving unit (18) for receiving (62) at least one radar wave that has been generated by reflection of the transmitted radar wave (16), - operating the signal processing and control unit (26) for o decomposing (64) the received at least one radar wave (22) into range, Doppler and angular information, o quantifying and tracking (68) a movement in each region of interest (38, 40, 42) of a plurality of predefined different regions of interest (38, 40, 42) by angular gating and range gating, o detecting (78) and monitoring vital signs of occupants (28, 30) in each region of interest (38, 40, 42), o for each region of interest (38, 40, 42), determining (84) whether quantified and tracked movements in the scene are related to an occupant (28, 30) or to external or internal disturbances, based on a fulfillment of at least one predefined condition concerning the quantified and tracked movements and/or the detected and monitored vital signs. ————"""""

2 The method as claimed in claim 1, wherein the step of determining (84) whether quantified and tracked movements in the scene are related to an occupant (28, 30) or to external or internal disturbances is based on a temporal development of fulfillments of the at least one predefined condition concerning the quantified and tracked movements and the detected and monitored vital signs.

3. The method as claimed in claim 1 or 2, wherein the at least one predefined condition concerning the quantified and tracked movements and the detected and monitored vital signs includes consideration of a constraint regarding a number of passengers (28, 30) in the vehicle interior (34) to be constant while the vehicle (32) is driving.

4 The method as claimed in any one of the preceding claims, wherein the step of decomposing (64) the received radar waves (22) into range, Doppler and angular information includes applying a fast Fourier transform to an intermediate frequency signal of the received radar waves (22) for converting the intermediate frequency signal of the received radar waves (22) from the time domain into the range domain.

5. The method as claimed in claim 4, wherein the step of decomposing (64) the received radar waves (22) further includes removing a static scene from the | 20 fast Fourier transform of the intermediate frequency signal of the received radar waves (22) by making use of a fast Fourier transform of an intermediate frequency signal of at least one previously received radar waves (22).

6. The method as claimed in any one of the preceding claims, wherein the step of quantifying and tracking (68) a movement in each region of interest (38, 40, 42) includes comparing an amount representing the quantified movement with at least one predefined detection threshold value (threat, thTshore, thTiong) for the quantity.

7 The method as claimed in any one of the preceding claims, wherein the at least one predefined detection threshold value (thr,eat» tATsnort» thTiong) for the quantity is a function of at least one out of vehicle speed and vehicle acceleration. ee —

8. The method as claimed in any one of the preceding claims, wherein the step of detecting (78) and monitoring vital signs of occupants (28, 30) comprises calculating (80) a phase signal from the fast Fourier transform of IF signals of the received radar waves (22) at determined movements in each region of interest (38, 40, 42), and processing each phase signal by reconstructing (82) the frequency domain signal into a phase-coherent time domain signal.

9 The method as claimed in any one of the preceding claims, wherein the step of quantifying and tracking (68) a movement in each region of interest (38, 40, 42) includes calculating power integrals in predefined regions for each region of interest (38, 40, 42).

10. The method as claimed in any one of the preceding claims, wherein the plurality of predefined different regions of interest (38, 40, 42) includes at least one out of a range-azimuth, range-elevation or azimuth-elevation region of interest.

11. The method as claimed in any one of the preceding claims, comprising a step of operating the signal processing and control unit for generating (86) an output signal that is indicative of the determined occupancy of the vehicle interior (34).

12. A radar sensor system (10) for detecting an occupancy in an interior (34) of a vehicle (32) and for vital sign monitoring, the radar sensor system (10) comprising - a radar transmitting unit (12) being configured for transmitting radar waves (16) towards a scene within the vehicle interior (34), - a radar receiving unit (18) being configured for receiving radar waves (22) that have been transmitted by the radar transmitting unit (12) and have been reflected by an object within the scene, and - a signal processing and control unit (26) that is at least configured to derive range information, Doppler information and angle-of-arrival information from the received radar waves (22), to control operation of the radar transmitting unit (12) and the radar receiving unit (18) and to automatically execute steps of the method as claimed in any one of the preceding claims. ——_—

13. The radar sensor system (10) as claimed in claim 12, wherein the radar transmitting unit (12) includes a plurality of radar transmitting antennas (14) and the radar receiving unit (18) includes a plurality of radar receiving antennas (20), and the radar transmitting antennas (14) and the radar receiving antennas (20) are configured to operate in a multiple-input and multiple-output configuration.

14. The radar sensor system (10) as claimed in claim 12 or 13, wherein the radar transmitting unit (12) is configured to transmit frequency-modulated continuous radar waves.

15. A software module (36) for controlling automatic execution of the method as claimed in any one of claims 1 to 11, wherein method steps to be conducted are converted into a program code of the software module (36), wherein the program code is implementable in a digital data memory unit of the radar sensor system (10) or a separate control unit and is executable by a processor unit of the radar sensor system (10) or a separate control unit.

EE HK