Classifications

• • G—PHYSICS
  • G08—SIGNALLING
  • G08B—SIGNALLING SYSTEMS, e.g. PERSONAL CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
  • G08B13/00—Burglar, theft or intruder alarms
  • G08B13/18—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength
  • G08B13/189—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems
  • G08B13/194—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
  • G08B13/196—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
  • G08B13/19639—Details of the system layout
  • G08B13/19647—Systems specially adapted for intrusion detection in or around a vehicle
• • G—PHYSICS
  • G08—SIGNALLING
  • G08B—SIGNALLING SYSTEMS, e.g. PERSONAL CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
  • G08B13/00—Burglar, theft or intruder alarms
  • G08B13/18—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength
  • G08B13/189—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems
  • G08B13/194—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
  • G08B13/196—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
  • G08B13/19639—Details of the system layout
  • G08B13/19652—Systems using zones in a single scene defined for different treatment, e.g. outer zone gives pre-alarm, inner zone gives alarm
• • G—PHYSICS
  • G08—SIGNALLING
  • G08B—SIGNALLING SYSTEMS, e.g. PERSONAL CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
  • G08B13/00—Burglar, theft or intruder alarms
  • G08B13/18—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength
  • G08B13/189—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems
  • G08B13/194—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
  • G08B13/196—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
  • G08B13/19602—Image analysis to detect motion of the intruder, e.g. by frame subtraction
  • G08B13/19606—Discriminating between target movement or movement in an area of interest and other non-signicative movements, e.g. target movements induced by camera shake or movements of pets, falling leaves, rotating fan

Definitions

• the present disclosure is directed to a computer-implemented method and a device for monitoring an environment of a motor vehicle.
• a computer program and a computer-readable medium are provided.
• CN115880634A is directed to a vehicle monitoring method and device and a computer readable storage medium and relates to the technical field of vehicle monitoring.
• the vehicle monitoring method comprises the following steps: obtain a current surrounding image of the vehicle; identifying a target person in the current surrounding image, and judging whether the target person is located in a preset vigilance area or not; if the target person is located in the preset vigilance area, the staying duration of the target person in the preset vigilance area is counted; and when the stay duration reaches a preset duration threshold, executing a preset alert operation.
• one object of the present disclosure may be to provide a method and/or a device configured to monitor an environment of a motor vehicle and trigger a control signal based on a behavior of a person being located in the environment, each of which is suitable for enriching the state of the art.
• the object is solved by a computer-implemented method, wherein the method comprises receiving sensor data corresponding to an area in an environment of a motor vehicle.
• the area is divided into at least two sub areas, wherein a first one of the at least two sub areas is located closer to the motor vehicle than a second one of the at least two sub areas.
• the method comprises detecting and tracking a person while being located in the first and/or the second sub area using the received sensor data.
• the method comprises determining a first timespan during which the detected person is located within the first sub area and determining a second timespan during which the detected person is located within the second sub area.
• the method comprises to output a control signal to the motor vehicle and/or an external data processing device when a sum of the determined first timespan and the second timespan exceeds a predefined first threshold and/or when the determined first timespan exceeds a predefined second threshold.
• a computer-implemented method may be a method where at least one part of the method is carried out by a computer.
• the computer may comprise hardware and/or software elements being designed/configured to carry out the at least one part of the method.
• the motor vehicle may be a personal vehicle, e.g., a car, and/or a commercial vehicle.
• the commercial vehicle may be a truck or lorry.
• the commercial vehicle may be a motor vehicle designed to transport freight, carry specialized payloads, and/or perform other utilitarian work.
• Commercial vehicles may vary greatly in size, power, and configuration, but may feature a body-on-frame construction, with a cabin that may be independent of the payload portion of the vehicle. Smaller varieties, e.g. pickup trucks, may be mechanically similar to some automobiles.
• Commercial vehicles may be configured to be mounted with specialized equipment, as is the case with refuse trucks, fire trucks, concrete mixers, and suction excavators.
• a commercial vehicle without a trailer may be called a "straight truck” while one designed specifically to pull a trailer may be called a “tractor”. Both, trucks and tractors, are referred to when the disclosure refers to a commercial vehicle. Moreover, the term "commercial vehicle” as used herein also refers to truck trailer systems.
• a sensor system capturing the sensor data may be installed at the motor vehicle, and may comprise at least one of the following sensors: a camera, a radar sensor, a lidar sensor, and/or an infrared sensor.
• a method may detect and track people in a series of control zones around a motor vehicle. If a person stays in a control zone exceeding a predefined time limit, the method may trigger a control signal, e.g., triggering a notification to driver and/or an operation center.
• the severity of the notification triggered by the control signal may depend on the sub area and thus the distance of the person to the motor vehicle.
• An identifier may be defined as a name that identifies (in other words, labels the identity of) a unique object, whereas the "object" is the detected person.
• the identifier may be a word, number, letter, symbol, or any combination of those.
• the words, numbers, letters, or symbols may follow an encoding system (wherein letters, digits, words, or symbols stand for [represent] ideas or longer names) or they may be arbitrary.
• the identifier may be a unique identifier.
• the unique identifier is an identifier that refers to only one instance, i.e., only one particular object in the environment of the motor vehicle.
• identifier allows the tracking of a detected person within the whole area, and thus over multiple sub-areas. Therefore, situations may be avoided in which the timer or counter for one detected person is reset when leaving one sub-area and entering another sub-area.
• the method may comprise determining if the detected person is a driver of the motor vehicle, wherein carrying out the method is stopped for the person that is determined to be the driver and/or for a certain area around the person that is determined to be the driver.
• a driver presence detection function may be implemented.
• the function may detect driver presence, for example, by using a vehicle key, a mobile device of the driver, an RFID Tag, an UWB Tag and/or any other kind of detection system that can identify the driver location with a reasonable accuracy. If a driver presence is detected by the driver presence detection function the tracking of the driver may be stopped and/or tracking of any other person within a certain area around the driver, e.g. on the same side of the motor vehicle as the driver, may be stopped.
• a predefined first notification may be associated with the first sub-area, wherein the control signal output when the sum of the determined first timespan and the second timespan exceeds the predefined first threshold is configured to trigger the first notification.
• a predefined second notification may be associated with the second sub-area, wherein the control signal output when the second timespan exceeds the predefined second threshold is configured to trigger the second notification.
• the severity of the notification triggered by the control signal may depend on the sub-area and thus the distance of the person to the motor vehicle.
• the method may comprise capturing video data of the detected person, optionally using an RGB (red-green-blue) camera installed at (optionally in) the motor vehicle.
• RGB red-green-blue
• Carrying out the method may be stopped when a predefined external control signal is received from a predefined source.
• the predefined source may comprise a door status determination device configured to determine a current opening status of a door (i.e., whether it is open or not) of the motor vehicle, and output the external control signal depending on the determined current opening status of the door, a human machine interface configured to receive input from a driver of the motor vehicle and to output the external control signal depending on the received input, and/or a fleet management system configured to output the external control signal.
• a door status determination device configured to determine a current opening status of a door (i.e., whether it is open or not) of the motor vehicle, and output the external control signal depending on the determined current opening status of the door
• a human machine interface configured to receive input from a driver of the motor vehicle and to output the external control signal depending on the received input
• a fleet management system configured to output the external control signal.
• the method may comprise receiving a location information representing the current location of the motor vehicle, wherein carrying out the method is stopped when the current location is outside a predefined area.
• the monitoring function may always be activated when the vehicle is stopped (with and without the ignition turned on).
• the monitoring function may be deactivated under certain conditions to avoid false alarms (optionally issued either to the driver or to the driver and the fleet management). To do so, the monitoring function may be deactivated: (1) fleet management triggered, (2) geofence triggered, (3) driver triggered, and/or (4) door status triggered.
• the method may comprise determining a body pose of the detected person (optionally based on the sensor data), wherein only those timespans are taken into account when determining the first and/or the second timespan during which the determined body pose fulfills a predefined criteria.
• a time counting system based on a body pose estimation may be implemented.
• the body pose estimation can be used for example to detect a relative pose of the detected person in relation to the vehicle and determine whether the detected person is facing away from the vehicle or not. If the detected person is facing away from vehicle, it is unlikely that the detected person poses an immediate risk, and the time should not be counted. However, if the detected person is facing in any other direction, it should be interpreted as a potential risk and the time should be counted.
• the above described method may be summarized with other words and with respect to a more concrete implementation thereof, as outlined below.
• the method or function (which may be implemented in soft- and/or hardware) may be configured to detect and track people in a series of control zones around a vehicle. If a person remains in a control zone for longer than a predefined time limit, the system may trigger a notification to a driver and/or an operation center. The severity of the notification may be based on the distance and/or behavior of the person. More specifically, when a person enters a control zone for the first time, the function may attribute an ID to the person, start counting time, and record the zone that the person entered.
• the function may stop monitoring. If the person remains in the control zone for longer than the time limit, the function may trigger a notification to the driver and/or the fleet management according to a (preset or predefined) notification level.
• a level 1 notification may be triggered if the person continues walking and/or stopping nearby, i. e. within a third control zone, for longer than five minutes.
• a level 2 notification may be triggered if the person walks or stops next to the vehicle without making contact, and remaining in a second control zone for more than two minutes.
• a level 3 notification may be triggered if the person walks or stops next to the vehicle without making contact, and remaining in a first control zone for more than 60 seconds.
• the first control zone is the one directly next to the vehicle, followed by the second control zone which is in turn followed by the third control zone such that the second control zone is sandwiched between the first and the third control zone.
• the function may keep monitoring and using the original or initially assigned ID that was attributed to the person when they first entered a control zone.
• the notification may be triggered when the time spent in the current zone exceeds the maximum time of that zone, or the total time of being present in any of the zones exceeds the maximum time defined for the third control zone.
• the system may directly trigger a level 3 notification level without resetting the timer.
• the function may automatically activate an RGB camera and begin recording and/or streaming the footage. If auxiliary lights are available, those lights may be activated as well to ensure high-quality recording of the event.
• the recorded event can be displayed on the driver's display, streamed via telematics, and/or viewed on the driver's mobile application.
• the above described function may be able to track multiple people simultaneously.
• the function may be able to detect when a person leaves a control zone and enters another zone.
• the function may be able to generate accurate and timely notifications.
• the function may be able to operate in different lighting conditions. For example, the function may be able to work at night or in low-light conditions.
• This function may offer a number of benefits, e.g., proactive intrusion detection (i.e., the system can detect potential intrusions before they occur, giving the driver and/or telematic system time to respond), high accuracy (i.e., an artificial intelligence perception software can accurately distinguish between humans and other objects, reducing the risk of false alarms), customizable warnings (i.e., the function can be configured to trigger warnings based on the severity of the intrusion, allowing the driver and/or telematic system to respond appropriately), and visual evidence (i.e., an RGB camera can provide visual evidence of intrusions, which can be used to identify and apprehend suspected criminals).
• proactive intrusion detection i.e., the system can detect potential intrusions before they occur, giving the driver and/or telematic system time to respond
• high accuracy i.e., an artificial intelligence perception software can accurately distinguish between humans and other objects, reducing the risk of false alarms
• customizable warnings i.e., the function can be configured to trigger warnings based on
• a data processing device comprising a processor being configured to carry out the method described above.
• the data processing device may be a control device or control unit and can be part of a driver assistance system or represent it.
• the control device may be, for example, an electronic control unit (ECU).
• the ECU may be an intelligent processor-controlled unit that can communicate with other modules via a central gateway (CGW), which can, for example, form the vehicle electrical system via field buses such as the CAN bus, LIN bus, MOST bus, FlexRay and/or via automotive Ethernet, e.g., together with telematics control units and/or an environment sensor system.
• CGW central gateway
• the data processing device controls functions relevant to the driving behavior of the vehicle, such as steering, engine control, power transmission and/or the braking system.
• driver assistance systems such as a parking assistant, adaptive cruise control (ACC), a lane departure warning system, a lane change assistant, traffic sign recognition, light signal recognition, a collision assistant, a night vision assistant, an intersection assistant, and/or an automated emergency braking system (AEBS) may be controlled by the control device.
• ACC adaptive cruise control
• a lane departure warning system e.g., a lane departure warning system
• a lane change assistant e.g., traffic sign recognition, light signal recognition, a collision assistant, a night vision assistant, an intersection assistant, and/or an automated emergency braking system (AEBS)
• AEBS automated emergency braking system
• a motor vehicle comprising the data processing device described above, and sensor system being configured to capture the sensor data corresponding to the area in the environment of the motor vehicle.
• the motor vehicle can be automated.
• the motor vehicle can be designed to at least partially and/or at least temporarily take over longitudinal guidance and/or lateral guidance during automated driving of the vehicle by means of the control device.
• Automated driving can take place in such a way that the movement of the vehicle is (largely) autonomous. Automated driving can be at least partially and/or temporarily controlled by the control device.
• the commercial vehicle can be an autonomous level 1 vehicle, i.e. it can have certain driver assistance systems that support the driver in operating the vehicle, such as adaptive cruise control (ACC).
• ACC adaptive cruise control
• the commercial vehicle can be a motor vehicle of autonomy level 2, i.e. be partially automated in such a way, that functions such as automatic parking, lane keeping or lateral guidance, general longitudinal guidance, acceleration and/or braking are taken over by driver assistance systems.
• the commercial vehicle can be an autonomous level 3 vehicle, i.e. conditionally automated in such a way, that the driver does not have to continuously monitor the vehicle system.
• the vehicle independently performs functions such as activating the indicator, changing lanes, and/or keeping the lane.
• the driver can turn their attention to other things, but is prompted by the system to take the lead within a warning time if necessary.
• the commercial vehicle can be an autonomous level 4 vehicle, i.e. highly automated, to such an extent, that the vehicle is permanently guided by the vehicle system. If the system can no longer fulfill the driving tasks, the driver can be requested to take over the driving.
• autonomous level 4 vehicle i.e. highly automated, to such an extent, that the vehicle is permanently guided by the vehicle system. If the system can no longer fulfill the driving tasks, the driver can be requested to take over the driving.
• a computer program comprising instructions which, when the program is executed by a computer, cause the computer to carry out the method described above may be provided.
• a program code of the computer program can be available in any code, in particular in a code that is suitable for motor vehicle control systems.
• a data carrier signal carrying the computer program may be provided.
• a computer-readable (storage) medium comprising instructions which, when executed by a computer, cause the computer to carry out the method described above may be provided.
• a computer-readable medium comprising a computer program as defined above can be provided.
• the computer-readable medium can be any digital data storage device, such as a USB flash drive, a hard drive disk, a CD-ROM, an SD card or an SSD card.
• the computer program does not necessarily have to be stored on such a computer-readable storage medium in order to be made available to the commercial vehicle, but can also be obtained via the Internet or otherwise externally using data transmission/data carrier signal(s).
• FIG. 1 a motor vehicle 1, here a truck trailer system comprising a truck 11 and a trailer 12 connected to the truck 11, is shown schematically in a bird's-eye perspective.
• a coordinate system is shown with an X axis extending along a longitudinal direction of the vehicle, a Y axis extending along a width direction of the vehicle and a Z axis which is parallel to a vertical direction.
• the truck comprises side view mirrors 13, wherein a sensor system 2 is installed in both of the side view mirrors 13. For reasons of clarity, only one of the two sensor systems 2 is and described shown.
• the sensor system 2 comprises an infrared sensor 21 and an RGB camera 22 with an overlapping field of views 23, 24 along the side of the truck 11 and the trailer 22 where the sensor system 2 is installed.
• an area 3 which comprises three sub-areas or zones 31 - 33.
• a first one of the sub-areas i.e., the first sub area 31, is located right next to the truck 11 and the trailer 12, has an essentially rectangular shape and extends 0,5 m in Y, away from the side of the truck 11 and the trailer 12, as well as 0,5 m in X, away from the end of the trailer 12.
• a second one of the sub-areas i.e. the second sub area 32, is located right next to the truck 11 and the trailer 12, has an essentially rectangular shape and extends 1 m in Y, away from the side of the truck 11 and the trailer 12 as well as 1 m in X, away from the end of the trailer 12.
• a third one of the sub-areas i.e. the third sub area 33, is located right next to the truck 11 and the trailer 12, has a substantially rectangular shape and extends 2 m in Y, away from the side of the truck 11 and the trailer 12 as well as 2 m in X, away from the end of the trailer 12.
• the first sub-area 31 is thus located closest to the truck 11 and the trailer 12 in Y, followed by the second sub area 32, and then the third sub area 33.
• sensor data corresponding to the area 3 is received at the data processing device 4, the sensor data being captured by the camera system 2.
• a second step 102 of the method 1000 the person and its presence in one of the sub-areas 31 - 33 is detected using the received sensor data.
• a third step 103 of the method 1000 it is determined if the detected person is a driver of the motor vehicle 1, wherein carrying out the method 1000 is stopped for the person that is determined to be the driver and/or for a certain area around the person that is determined to be the driver. More specifically, in this present embodiment monitoring of the side of the motor vehicle 1 is stopped on which the driver is detected. If the detected person is not the driver, a fourth step 104 of the method 1000 is carried out.
• tracking of the detected person is started and carried out as long as the detected person is located in one of the sub-areas 31 - 33.
• the received sensor data is used for the tracking. Tracking the detected persons comprises determining in which sub area 31 - 33 the detected person is located as well as determining a body pose of the detected person.
• the camera 22 is used for capturing and storing video data of the detected person in a fifth step 105 for a as long as the detected person stays or remains within the first sub area 31.
• step 104 If it is determined in the fourth step 104 that the detected person is currently located within the first sub area 31, it is determined based on the received sensor data if the detected person touches and/or tries to enter the motor vehicle 1 in a sixth step 106. If is determined in the sixth step 106 that the detected persons touches and/or tries to enter the motor vehicle 1, a control signal is output in a seventh step 107 to the motor vehicle 1 and/or an external data processing device, the control signal triggering a level 3 notification.
• step 108 of the method 1000 it is determined whether the detected person is entering the sub area 31 - 33 where the person is currently located from another sub area 31 - 33 or if the detected person is initially entering the area 3, i.e., a first entering. If the person is initially entering the area 3, a ninth step 109 is carried out. Otherwise, a tenth step 110 is carried out directly.
• an identifier is assigned to the detected person that stays with the detected person until the person leaves the area 3.
• the identifier is used for the tracking of the detected person initiated in the fourth step 104 of the method.
• a total timespan is determined during which the detected person was located within one of the sub-areas 31 - 33 and also the respective timespans for all of the sub-areas 31 - 33 are determined, i.e. a first timespan during which the detected person is located within the first sub area 31, a second timespan during which the detected person is located within the second sub area 32, and a third timespan during which the detected person is located within the third sub area 33.
• the total timespan may be determined by calculating the sum of the first, the second and the third timespan. In determining the timespans only those timespans are taken into account during which the determined body pose (see step 104) fulfills a predefined criterion, e.g. where the detected person faces the motor vehicle 1.
• an eleventh step 111 of the method 1000 it is determined whether the determined total timespan exceeds a predefined third threshold, i.e., the threshold assigned to the third sub area 33.
• a predefined third threshold i.e., the threshold assigned to the third sub area 33.
• the eleventh step 111 of the method 1000 it is also determined if the determined first timespan exceeds a predefined first threshold, i.e., the threshold assigned to the first sub area 31, and if the determined second timespan exceeds a predefined second threshold, i.e., the threshold assigned to the second sub area 32.
• a control signal is output in a twelves step 112 to the motor vehicle 1 and/or an external data processing device, the control signal triggering a level 1 notification.
• a control signal is output in a thirteenth step 113 to the motor vehicle 1 and/or an external data processing device, the control signal triggering a level 3 notification.
• a so-called level 1 notification may be triggered because of incidents inside any of the three sub-areas 31 - 33.
• An incident may be when a person remains walking and/or stopping within the area 3 for longer than 5 minutes.
• a so-called level 2 notification may be triggered because of incidents inside the second sub area 32.
• An incident may be when a person walks and/or stops beside the vehicle 1 remaining in sub-area 32 for more than 2 minutes but without making contact with the vehicle 1.
• a so-called level 3 notification may be triggered because of incidents inside the first sub-area 31.
• the initial step 100 comprises determining if a predefined condition is met leading to a stop of the method 1000.
• the predefined condition is met when a predefined external control signal is received from a predefined source.
• the predefined source comprises a door status determination device configured to determine a current opening status of a door of the motor vehicle, and outputs the external control signal depending on the determined current opening status of the door, a human machine interface configured to receive input from a driver of the motor vehicle and to output the external control signal depending on the received input, and/or a fleet management system configured to output the external control signal.
• the predefined condition is also met when a location information representing the current location of the motor vehicle is received, wherein said received current location is outside a predefined area.

Landscapes

• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• Traffic Control Systems (AREA)

Priority Applications (1)

Applications Claiming Priority (1)

Publications (1)

Family

ID=91248494

Family Applications (1)

Country Status (1)

Citations (4)

• 2024
  • 2024-05-23 EP EP24177637.6A patent/EP4654166A1/de active Pending

Patent Citations (4)

Similar Documents

Legal Events