WO2024121876A1 - 置き去り判定システム、置き去り判定装置、および、置き去り判定方法 - Google Patents

置き去り判定システム、置き去り判定装置、および、置き去り判定方法 Download PDF

Info

Publication number
WO2024121876A1
WO2024121876A1 PCT/JP2022/044646 JP2022044646W WO2024121876A1 WO 2024121876 A1 WO2024121876 A1 WO 2024121876A1 JP 2022044646 W JP2022044646 W JP 2022044646W WO 2024121876 A1 WO2024121876 A1 WO 2024121876A1
Authority
WO
WIPO (PCT)
Prior art keywords
determination
vehicle
abandonment
vehicle interior
unit
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2022/044646
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
靖憲 星原
勲旺 松島
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Mitsubishi Electric Corp
Original Assignee
Mitsubishi Electric Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Mitsubishi Electric Corp filed Critical Mitsubishi Electric Corp
Priority to JP2024562388A priority Critical patent/JP7778250B2/ja
Priority to PCT/JP2022/044646 priority patent/WO2024121876A1/ja
Publication of WO2024121876A1 publication Critical patent/WO2024121876A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
    • G01S13/02—Systems using reflection of radio waves, e.g. primary radar systems; Analogous systems
    • G01S13/50—Systems of measurement based on relative movement of target
    • G01S13/52—Discriminating between fixed and moving objects or between objects moving at different speeds
    • G01S13/56—Discriminating between fixed and moving objects or between objects moving at different speeds for presence detection
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
    • G01S13/88—Radar or analogous systems specially adapted for specific applications
    • G01S13/89—Radar or analogous systems specially adapted for specific applications for mapping or imaging
    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING SYSTEMS, e.g. PERSONAL CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING SYSTEMS, e.g. PERSONAL CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
    • G08B21/18—Status alarms
    • G08B21/22—Status alarms responsive to presence or absence of persons

Definitions

  • This disclosure relates to an abandonment determination system, an abandonment determination device, and an abandonment determination method.
  • a technology uses a biosensor that detects the biological signals of an infant or other person present in the vehicle cabin from movements on the surface of the body as a sensor for determining whether or not an infant or other person has been left behind in the vehicle cabin, and determines whether or not an infant or other person has been left behind in the vehicle cabin based on the biological information from the biosensor (for example, Patent Document 1).
  • the radio waves emitted by the radio wave sensor have the property of being transmitted, reflected, or diffracted depending on the materials of the structures inside the vehicle cabin, so the radio wave sensor can detect the position and movement of moving objects inside the vehicle cabin over a wide area. Therefore, by determining whether or not a child or other person has been left behind inside the vehicle cabin based on information generated from the reflected waves received by the radio wave sensor, it becomes possible to determine whether or not a child or other person has been left behind inside the vehicle cabin over a wide area.
  • radio waves emitted by a radio wave sensor installed inside the vehicle cabin are propagated outside the vehicle cabin, which is not originally intended.
  • a radio wave sensor installed inside the vehicle cabin can also detect people and other objects present outside the vehicle cabin. Therefore, due to the radio wave sensor's properties of transmission, reflection, or diffraction, there is a possibility that the radio wave sensor may determine that a person or other object present outside the vehicle cabin is actually present inside the vehicle cabin.
  • the technology for determining whether or not a child has been left behind using a biosensor does not take into account the possibility of erroneous determination of whether or not a child has been left behind due to the nature of the radio wave sensor, as described above, and therefore remains unable to solve the above problem.
  • the present disclosure has been made to solve the above problems, and aims to provide an abandonment determination system that prevents erroneous determination of whether or not a child or other person has been left behind in a vehicle.
  • the abandoned person determination system includes a radio wave sensor that is provided in the vehicle interior and detects moving objects based on radio waves radiated toward the vehicle interior and reflected by objects in the vehicle interior, a biometric information acquisition unit that acquires biometric information extracted based on the detection result of the moving object detected by the radio wave sensor, a three-dimensional image acquisition unit that acquires a three-dimensional image that shows in three dimensions the distribution of the area in the vehicle interior where the moving object is present, generated based on the detection result of the moving object detected by the radio wave sensor, an interior vehicle determination unit that performs an interior vehicle interior determination as to whether or not a moving object deemed to be a person requiring assistance is present in the vehicle interior based on the three-dimensional image acquired by the three-dimensional image acquisition unit and the biometric information acquired by the biometric information acquisition unit, an exterior vehicle interior determination unit that performs an exterior vehicle interior determination as to whether or not a moving object deemed to be a person is present outside the vehicle interior based on the three-dimensional image acquired by the three-
  • This disclosure makes it possible to prevent erroneous determinations of whether or not a child or other person has been left behind in the vehicle.
  • FIG. 1 is a diagram showing a configuration example of an abandonment determination system according to a first embodiment
  • 1 is a diagram showing a configuration example of a biometric determination device according to a first embodiment
  • 1 is a diagram illustrating an example of the configuration of an abandonment determination device according to a first embodiment
  • 1 is a diagram for explaining in detail the radio wave sensor according to the first embodiment
  • 1 is a diagram for explaining in detail the radio wave sensor according to the first embodiment
  • 4 is a diagram for explaining a flow of signals in a radio wave transmitting/receiving section of the radio wave sensor according to the first embodiment.
  • FIG. FIG. 7A is an explanatory diagram showing an example of an FM transmission wave and an FM reception wave
  • FIG. 7B is an explanatory diagram showing the relationship between the transmission and reception period of the FM transmission wave and the frequency difference fd .
  • 8A, 8B, and 8C are diagrams showing examples of a three-dimensional image generated by the three-dimensional image generating unit in the first embodiment.
  • 1A and 1B are diagrams illustrating Doppler components when a moving object (occupant) is present and when a moving object (occupant) is not present;
  • 13 is a diagram showing an example of a first table for abandonment determination used by a first determination unit in a first abandonment determination process in the first embodiment;
  • FIG. 13 is a diagram showing an example of a second table for abandonment determination used by the second determination unit in the second abandonment determination process in embodiment 1.
  • FIGS. 12A and 12B are diagrams showing examples of a first alarm table and a second alarm table used by the alarm control unit to determine whether to output an alarm instruction in embodiment 1, where FIG. 12A is a diagram showing an example of the first alarm table and FIG. 12B is a diagram showing an example of the second alarm table.
  • 4 is a flowchart for explaining the operation of the abandonment determination system according to the first embodiment.
  • 14 is a flowchart for explaining the process of step ST2 in FIG. 13 in detail.
  • 4 is a flowchart for explaining the operation of the alarm control device according to the first embodiment.
  • 16A and 16B are diagrams illustrating an example of a hardware configuration of the abandonment determination device according to the first embodiment.
  • passenger monitoring systems that detect passengers in the vehicle cabin using sensors and determine their behavior have been introduced to the market.
  • Representative functions of such passenger monitoring systems include, for example, a function to detect driver behavior such as dozing or looking away while driving, with the main purpose of driving assistance.
  • technological and product development of passenger monitoring systems to further improve vehicle safety is progressing, such as a seat belt reminder for all seats by having a function to detect the seating positions or physiques of all passengers in the vehicle cabin, or a system to determine whether or not a child has been left behind in the vehicle cabin.
  • leaving young children behind in vehicles has become a social issue, and Euro NCAP (New Car Assessment Programme), which handles automobile assessments in Europe, plans to add "Child Presence Detection" as a test item in 2022.
  • a camera is one of the sensors that is put to practical use in the vehicle interior.
  • the images captured by the camera can not only detect the presence or absence of an occupant in the vehicle cabin or determine the physique of the occupant, but also enable more advanced detection or determination, such as personal authentication through facial recognition or detection of drowsiness by checking whether the driver's eyes are open or closed.
  • detection of the presence or absence of an occupant in the vehicle cabin or determination of the occupant's physical size is performed using an image captured within the camera's visible range, and therefore the detection range or determination range is limited.
  • the radio wave sensor generates a three-dimensional image by receiving the reflected radio waves that are reflected by a target object (more specifically, an occupant) and processing the signals.
  • the three-dimensional image makes it possible to detect the presence or absence of an occupant in the vehicle cabin, determine the seating position of the occupant, determine the physique of the occupant, or detect the movement of the occupant.
  • the radio wave sensor also extracts minute movements, such as the breathing of the occupant, from the reflected waves as biometric information, which makes it possible to determine the presence or absence of an object (more specifically, an occupant).
  • radio wave irradiation which is the detection range of the radio wave sensor.
  • the radio waves emitted by a radio wave sensor have the property of being transmitted, reflected, or diffracted depending on the materials of the structures in the vehicle interior, such as the seats or ceiling (hereinafter referred to as "vehicle structures"). Therefore, by widening the antenna directivity of the radio wave sensor and optimizing the installation position, the radio wave sensor has the characteristic of being able to realize a wide detection range.
  • the antenna directivity of the radio wave sensor corresponds to the lens angle of the camera.
  • the radio wave sensor is capable of achieving a wide detection range.
  • radio waves emitted from a radio wave sensor installed in the vehicle interior to monitor passengers in the vehicle interior also propagate outside the vehicle interior, which is not originally intended. Therefore, the radio wave sensor detects people around the vehicle, in other words, people outside the vehicle interior. Details of the radio wave sensor detecting people outside the vehicle interior will be described later with reference to the drawings.
  • a radio wave sensor detects a person outside the vehicle and a determination is made as to whether a child or other person has been left behind inside the vehicle using a three-dimensional image or biometric information generated based on the result of the radio wave sensor detecting an object (more specifically, an occupant), it may be determined that a child or other person has been left behind inside the vehicle even though in fact they have not.
  • the present disclosure provides an abandonment determination system that can determine whether or not a person requiring assistance, such as a small child, has been left behind in a vehicle cabin over a wide range by using information from a radio wave sensor (more specifically, three-dimensional images and biometric information), and that prevents erroneous determination of whether or not a person requiring assistance has been left behind due to the properties of the radio wave sensor, as described above.
  • a radio wave sensor more specifically, three-dimensional images and biometric information
  • the term "person requiring assistance” refers to a living body, such as a small child, that has difficulty getting out of the vehicle on its own if left behind in the vehicle.
  • the term “person requiring assistance” also includes living bodies such as pets.
  • the "person requiring assistance” is an infant, and the abandonment determination system determines whether an infant has been left behind in the vehicle cabin.
  • FIG. 1 is a diagram showing an example of the configuration of an abandonment determination system 100 according to the first embodiment.
  • the abandonment determination system 100 is composed of a liveness determination device 130 and an abandonment determination device 140 .
  • the abandonment determination device 140 is mounted on a vehicle and is connected to the biometric determination device 130 , the alarm control device 150 , and the vehicle information device 40 .
  • the biometric determination device 130 is mounted on a radio wave sensor 11 provided in the vehicle interior.
  • the biometric determination device 130 determines whether or not a moving object deemed to be a child is present inside the vehicle cabin (hereinafter referred to as "inside vehicle cabin determination") and whether or not a moving object is present outside the vehicle cabin (hereinafter referred to as “outside vehicle cabin determination”) based on the biometric information generated based on the detection result of the moving object detected by the radio wave sensor 11 and the three-dimensional image.
  • “moving object” refers to a person, including a vehicle occupant.
  • the radio wave sensor 11 will be described in detail later.
  • 2 is a diagram showing a configuration example of the liveness determining device 130 according to embodiment 1. Details of the configuration example of the liveness determining device 130 will be described later.
  • the vehicle information device 40 is mounted on the vehicle and is capable of communicating with the short-range wireless communication device 45 .
  • the short-range wireless communication device 45 is, for example, a mobile terminal carried by a person such as a vehicle owner, a vehicle key having a so-called keyless entry function, or a so-called smart key.
  • the vehicle information device 40 plays a role of controlling the unlocking and locking of the vehicle doors by exchanging key information with the vehicle key, for example.
  • the vehicle information device 40 includes a wireless communication status monitoring unit 41.
  • the wireless communication status monitoring unit 41 monitors the communication status with the short-range wireless communication device 45. More specifically, the wireless communication status monitoring unit 41 monitors whether or not wireless communication between the vehicle and the short-range wireless communication device 45 has been established.
  • the wireless communication status monitoring unit 41 outputs information regarding the communication status between the vehicle and the short-range wireless communication device 45, i.e., whether or not wireless communication between the vehicle and the short-range wireless communication device 45 has been established (hereinafter referred to as "communication status information") to the abandoned vehicle determination device 140.
  • the abandonment determination device 140 determines whether or not a child has been left behind in the vehicle cabin based on the results of the interior and exterior determination performed by the biometric determination device 130 and the communication status information obtained from the vehicle information equipment 40.
  • 3 is a diagram showing a configuration example of the abandonment determination device 140 according to embodiment 1. The configuration example of the abandonment determination device 140 will be described in detail later.
  • the alarm control device 150 is mounted on the vehicle and causes the alarm device 160 to output an alarm based on the result of the determination by the abandonment determination device 140 as to whether or not an infant has been left behind inside the vehicle compartment.
  • the warning device 160 is assumed to be, for example, a display device such as a display or an audio output device such as a speaker provided on a mobile terminal carried by the owner of the vehicle, or a lamp or horn provided in the vehicle. A detailed configuration example of the alarm control device 150 will be described later.
  • the radio wave sensor 11 will be described in detail.
  • the radio wave sensor 11 according to the first embodiment is equipped with an antenna having wide-angle directivity, and is provided inside the vehicle interior so that radio waves are irradiated to any occupants who may be present inside the vehicle interior.
  • FIG. 4 and 5 are diagrams for explaining the radio wave sensor 11 according to the first embodiment in detail.
  • FIG. 4 is a diagram showing an example of the interior of a vehicle cabin as viewed from the side of the vehicle.
  • FIG. 5 is a diagram showing an example of the interior and exterior of a vehicle when the vehicle is viewed from above.
  • the vehicle is designated "10".
  • the x-axis is an axis parallel to the vehicle width direction
  • the y-axis is an axis parallel to the vehicle height direction
  • the z-axis is an axis parallel to the vehicle length direction.
  • "parallel” is not limited to being strictly “parallel” and includes approximately parallel.
  • 4 and 5 are right-hand drive vehicles, this is merely an example, and the vehicle may be a left-hand drive vehicle.
  • the radio wave sensor 11 is installed, for example, at an upper portion inside the vehicle cabin.
  • FIG. 4 shows the state of the interior of the vehicle when there is an adult passenger 30 in the passenger seat and an occupant (more specifically, a child) 31 lying in a child seat 20 installed facing backwards in the rear seat.
  • the radio wave sensor 11 irradiates radio waves toward the occupants 30 and 31 inside the vehicle interior, and receives reflected waves of the radio waves from the occupants 30 and 31.
  • the image of the radio wave emitted from the radio wave sensor 11 and the reflected wave from the radio wave reflected by the occupant 30 and the occupant 31 is indicated by "13.”
  • FIG. 5 is a diagram showing the state inside and outside the vehicle cabin when an adult occupant 35 who was in the vehicle cabin gets out of the vehicle. There is no occupant in the vehicle cabin.
  • the occupant 35 is assumed to be, for example, a vehicle owner.
  • the occupant 35 carries a short-range wireless communication device 45 and moves between within the communication range of the short-range wireless communication device 45 (indicated by "46" in FIG. 5) and outside the communication range of the short-range wireless communication device 45.
  • FIG. 5 illustrates both the occupant 35 who is within the communication range of the short-range wireless communication device 45 and the occupant 35 who is outside the communication range of the short-range wireless communication device 45.
  • the radio wave sensor 11 is provided inside the vehicle cabin so that radio waves are irradiated to any occupants who may be present inside the vehicle cabin.
  • the irradiation range of the radio waves emitted by the radio wave sensor 11 is indicated by "15".
  • the irradiation range into which the radio wave sensor 11 emits radio waves can also be considered as the detection range in which the radio wave sensor 11 can detect an object.
  • the radio wave sensor 11 may process the reflected waves of the radio waves emitted to the people outside the vehicle cabin as signals of radio waves reflected by passengers inside the vehicle cabin.
  • the radio wave sensor 11 includes a radio wave transmitting/receiving unit 110, a sensor signal processing unit 120, and a liveness determination device 130.
  • the radio wave transmitting/receiving section 110 is made up of a transmitting antenna 111, a receiving antenna 112, a high frequency signal generating circuit section 113, a transmitting circuit section 114, a receiving circuit section 115, and an A/D conversion section .
  • the transmitting antenna 111 is a planar antenna configured on an electronic circuit board.
  • the transmitting antenna 111 has a plurality of transmitting antenna elements (not shown) that radiate radio waves toward a target area.
  • the receiving antenna 112 is a planar antenna configured on an electronic circuit board.
  • the receiving antenna 112 has a plurality of receiving antenna elements (not shown) that receive the reflected waves of the radio waves radiated from the transmitting antenna 111 .
  • FIG. 1 shows only one each of the transmitting antenna 111, receiving antenna 112, transmitting circuit unit 114, and receiving circuit unit 115, the transmitting antenna 111, receiving antenna 112, transmitting circuit unit 114, or receiving circuit unit 115 may be configured with multiple units in order to satisfy the requirements for signal processing performed by the sensor signal processing unit 120 (more specifically, the three-dimensional image generation unit 121 and the biometric information extraction unit 122).
  • the high frequency signal generating circuit section 113 generates a signal for sensing.
  • FIG. 6 is a diagram for explaining the flow of signals in the radio wave transmitting/receiving unit 110 of the radio wave sensor 11 according to the first embodiment.
  • modulation methods for the sensing signal of the radio wave sensor 11 There are various modulation methods for the sensing signal of the radio wave sensor 11, but here, we will explain an example in which the FM-CW (Frequency Modulation - Continuous Wave) method, which is often used for in-vehicle applications, is applied as the modulation method.
  • the FM-CW method is suitable for use in passenger monitoring systems because it can measure the distance, speed, and angle of a target object.
  • the high frequency signal generating circuit section 113 periodically generates an FM signal (called a chirp wave) whose frequency increases and decreases as shown in FIG. 6, and sends it to the transmitting circuit section 114 and the receiving circuit section 115.
  • an FM signal called a chirp wave
  • the transmission circuit unit 114 amplifies the signal power to obtain the power required for emitting radio waves, and emits the radio waves into the space inside the vehicle via the transmission antenna 111.
  • the target object refers to an object that reflects radio waves, such as the occupant 30 or occupant 31 shown in Figure 4, or a vehicle structure.
  • the receiving circuit section 115 receives, via the receiving antenna 112, the radio waves that have been reflected off the surface of the target object and returned to the radio wave transmitting/receiving section 110.
  • a signal similar to the FM transmission wave is input as a reception signal to the reception circuit section 115 as an FM reception wave.
  • the reception signal is input to the reception circuit section 115 with a time difference td that is required for the radio wave to reach the target object from the radio wave transmission/reception section 110 and return.
  • Fig. 7A is an explanatory diagram showing an example of the FM transmission wave and the FM reception wave.
  • the receiving circuit section 115 extracts the frequency difference fd between the frequency of the FM signal generated by the high frequency signal generating circuit section 113 and the frequency of the receiving signal, generates an intermediate frequency (IF) signal having the frequency difference fd , and outputs the intermediate frequency signal to the A/D (analog-digital) conversion section 116.
  • Fig. 7B is an explanatory diagram showing the relationship between the transmission and reception period of the FM transmission wave and the frequency difference fd .
  • the occupant 31 is located farther from the radio wave sensor 11 than the occupant 30. Therefore, when the occupant 31 is the target object, the time difference td and the frequency difference fd are larger than the time difference td and the frequency difference fd when the occupant 30 is the target object.
  • the A/D converter 116 converts the intermediate frequency signal from an analog signal to a digital signal.
  • the A/D conversion unit 116 outputs the digital signal to the sensor signal processing unit 120 .
  • the sensor signal processing unit 120 performs the following two types of signal processing (hereinafter referred to as “three-dimensional image generation processing” and “biometric information generation processing”).
  • the three-dimensional image generating process is performed by a three-dimensional image generating unit 121 included in the sensor signal processing unit 120.
  • the biometric information generating process is performed by a biometric information extracting unit 122 included in the sensor signal processing unit 120.
  • the three-dimensional image generating unit 121 generates a three-dimensional image based on the intermediate frequency signal output from the A/D conversion unit 116 of the radio wave transmitting/receiving unit 110 in a three-dimensional image generating process.
  • the intermediate frequency signal is generated from a reflected wave of a radio wave reflected by the surface of the target object.
  • the three-dimensional image generating unit 121 can calculate the position of the reflection point on the surface of the target object (more specifically, the distance and angle from the radio wave sensor 11) from the intermediate frequency signal.
  • the three-dimensional image generating section 121 can generate a three-dimensional image of the target object by performing signal processing on the reflected waves from the multiple reflection points.
  • the three-dimensional image generated by the three-dimensional image generating unit 121 represents in three dimensions the distribution of the area in the vehicle cabin where the moving object exists.
  • the three-dimensional image represents the minute movement of the moving object in the vehicle cabin, in other words, the target object that has reflected the radio wave irradiated by the radio wave sensor 11, by a plurality of grids that correspond to the reflection points of the radio wave in the three-dimensional space.
  • FIG. 8A, FIG. 8B, and FIG. 8C are diagrams showing examples of three-dimensional images generated by three-dimensional image generating section 121 in embodiment 1.
  • FIG. Fig. 8A shows an example of a top view of a three-dimensional image
  • Fig. 8B shows an example of a side view of a three-dimensional image
  • Fig. 8C shows an example of a front view of a three-dimensional image.
  • the three-dimensional image is shown in a three-view view for the sake of simplicity.
  • the three-dimensional images shown in Fig. 8A, Fig. 8B, and Fig. 8C are three-dimensional images in the case where the state of the interior of the vehicle is the state of the interior of the vehicle shown in Fig. 4.
  • a numerical value according to the speed of a target object existing at the position of the grid is assigned to each grid.
  • the faster the speed of a target object existing at the position of a certain grid the larger the numerical value assigned to that grid.
  • a grid included in a range in which a moving object with small movements exists is assigned a numerical value larger than that of a grid in which no moving object exists.
  • the grids to which larger numerical values are assigned are shown darker.
  • the grids included in the range in which a moving object exists are shown darker.
  • the occupant 31, i.e., the infant, is occluded by the child seat 20 or other vehicle structures. Due to this effect, the spatial distribution of the infant shown by the grid is hardly visible in the three-dimensional images shown in Fig. 8A, Fig. 8B, and Fig. 8C. In other words, it is difficult to determine the presence of an infant from the three-dimensional images shown in Fig. 8A, Fig. 8B, and Fig. 8C. As for the occupant 30, since the radio waves from the radio wave sensor 11 are irradiated onto the entire body of the occupant 30, the spatial distribution of the occupant 30 appears on the three-dimensional image as a grid based on the reflected waves.
  • the biometric information extraction unit 122 extracts, as biometric information, frequency components (Doppler components) superimposed on the reflected waves of a moving object due to minute movements such as breathing of the moving object, based on the intermediate frequency signal output from the A/D conversion unit 116 of the radio wave transmission/reception unit 110.
  • Fig. 9 is a diagram showing Doppler components when a moving object (occupant) is present and when no moving object (occupant) is present. Even if a spatial distribution indicating an infant does not appear on the three-dimensional image, as in the three-dimensional images shown in Figures 8A, 8B, and 8C, the biological information of the infant can be extracted by extracting the Doppler components.
  • the biometric information does not include information indicating the position of the moving object indicated by the biometric information, and therefore the position of the moving object cannot be identified from the biometric information. That is, even if it is attempted to determine the presence or absence of an occupant in the vehicle cabin from biometric information extracted based on the reflected waves of the radio waves irradiated by the radio wave sensor 11 reflected by the surface of the target object, it is difficult to distinguish between an occupant in the vehicle cabin and a person outside the vehicle cabin. For example, when the state of the inside and outside of the vehicle cabin is as shown in FIG. 5, the occupant 35 outside the vehicle cabin shown in FIG. 5 cannot be identified as a person outside the vehicle cabin from the biometric information.
  • the radio wave sensor 11 has a property that the radio waves irradiated by the radio wave sensor 11 are transmitted, reflected, or diffracted depending on the material of the vehicle structure. Due to this property, there is a possibility that the reflected waves that hit the surface of the person outside the vehicle cabin propagate into the vehicle cabin, and further hit the vehicle structure, and then the reflected waves are received by the radio wave sensor 11.
  • the biometric information extraction unit 122 may extract the biometric information of the person outside the vehicle cabin.
  • the sensor signal processing unit 120 outputs to the biometric determination device 130 the three-dimensional image generated by the three-dimensional image generation unit 121 in the three-dimensional image generation process and the biometric information extracted by the biometric information extraction unit 122 in the biometric information extraction process.
  • the biometric determination device 130 includes a biometric information acquisition unit 1301 , a three-dimensional image acquisition unit 1302 , an interior determination unit 131 , and an exterior determination unit 135 .
  • the vehicle interior determination unit 131 includes a first vehicle interior determination unit 132 and a second vehicle interior determination unit 133 .
  • the biometric information acquisition unit 1301 acquires the biometric information extracted by the biometric information extraction unit 122 of the sensor signal processing unit 120 .
  • the biometric information acquisition unit 1301 outputs the acquired biometric information to the second vehicle interior determination unit 133 of the vehicle interior determination unit 131.
  • the second vehicle interior determination unit 133 may have the function of the biometric information acquisition unit 1301.
  • the three-dimensional image acquisition unit 1302 acquires the three-dimensional image generated by the three-dimensional image generation unit 121 of the sensor signal processing unit 120 .
  • the three-dimensional image acquisition unit 1302 outputs the acquired three-dimensional image to the first vehicle interior determination unit 132 of the vehicle interior determination unit 131 and the vehicle exterior determination unit 135 .
  • the first vehicle interior determination unit 132 and the vehicle exterior determination unit 135 may have the functions of the three-dimensional image acquisition unit 1302.
  • the vehicle interior determination unit 131 determines whether or not a moving object regarded as a child is present inside the vehicle interior based on the three-dimensional image acquired by the three-dimensional image acquisition unit 1302 and the biometric information acquired by the biometric information acquisition unit 1301 . As described above, the vehicle interior determination unit 131 includes the first vehicle interior determination unit 132 and the second vehicle interior determination unit 133 .
  • the first vehicle interior determination unit 132 determines whether or not a moving object regarded as an infant is present in the vehicle interior (hereinafter referred to as "first vehicle interior determination") based on the three-dimensional image acquired by the three-dimensional image acquisition unit 1302. In the first embodiment, the process in which the first vehicle interior determination unit 132 performs the first vehicle interior determination is also referred to as “first vehicle interior determination process.” In the first embodiment, the first vehicle interior determination unit 132 determines whether or not "there is a moving object regarded as an infant in the vehicle interior” means, in more detail, whether or not "there is a moving object regarded as an abandoned infant in the vehicle interior.” In other words, the first vehicle interior determination unit 132 determines whether or not "there is only a moving object regarded as an infant in the vehicle interior.” The first vehicle interior determination process performed by the first vehicle interior determination unit 132 will be described in detail.
  • the first vehicle interior determination unit 132 first determines, in the part of the three-dimensional image corresponding to the area inside the vehicle interior, whether there is a moving object deemed to be an occupant inside the vehicle interior, and, if so, in which seat position the moving object deemed to be an occupant is located.
  • Each grid of the three-dimensional image is associated with a coordinate in the real space of the vehicle interior, which is expressed in a three-dimensional coordinate system that represents the real space of the vehicle interior.
  • the first vehicle interior determination unit 132 can identify an area of the three-dimensional image that corresponds to the area of the vehicle interior.
  • the three-dimensional image of the portion that corresponds to the area of the vehicle interior will be referred to as the "vehicle interior three-dimensional image.”
  • the first vehicle interior determination unit 132 determines whether or not a spatial distribution contained in the vehicle interior three-dimensional image (for example, the spatial distribution shown by a colored grid in Figure 8) is a spatial distribution corresponding to an occupant (hereinafter referred to as an ⁇ occupant spatial distribution'') based on the shape of the spatial distribution. Any method can be used for the judgment based on the shape of the spatial distribution, but one possible method is to use a model that has learned the shape of the occupant spatial distribution to judge whether the spatial distribution contained in a three-dimensional image of the interior of the vehicle is an occupant spatial distribution.
  • the first vehicle interior determination unit 132 may determine that the spatial distribution included in the vehicle interior three-dimensional image is an occupant spatial distribution if the relative speed vn corresponding to the spatial distribution included in the vehicle interior three-dimensional image is greater than a preset threshold value (hereinafter referred to as the "first speed determination threshold value"), for example.
  • a preset threshold value hereinafter referred to as the "first speed determination threshold value”
  • Each grid of the three-dimensional image is associated with the relative speed vn .
  • the first speed determination threshold value is stored in a location that can be referenced by the first vehicle interior determination unit 132. Even if the occupant is, for example, a small child lying in the child seat 20, there is a certain degree of movement. If the first vehicle interior determination unit 132 determines that the spatial distribution included in the vehicle interior three-dimensional image is an occupant spatial distribution, it determines that a moving object regarded as an occupant is present in the vehicle interior.
  • the first vehicle interior determination unit 132 determines that the spatial distribution included in the vehicle interior three-dimensional image is an occupant spatial distribution, it identifies the position of a moving object that is considered to be an occupant in the occupant spatial distribution from the distance R n , incidence angle ⁇ x,n and incidence angle ⁇ y,n related to the occupant spatial distribution. Note that if the distance R n from the radio wave sensor 11 and the incidence angle ⁇ x,n and incidence angle ⁇ y,n with respect to the radio wave sensor 11 are known, the first vehicle interior determination unit 132 can identify the position of the moving object that is considered to be an occupant.
  • the distance R n from the radio wave sensor 11 and the incidence angle ⁇ x,n and incidence angle ⁇ y,n with respect to the radio wave sensor 11 are associated with each grid of the vehicle interior three-dimensional image.
  • the first interior determination unit 132 may determine that the moving object to be regarded as an occupant is not present, for example, if the position of the moving object to be regarded as an occupant is a position that is not expected to be a position where an occupant is present.
  • the first vehicle interior determination unit 132 determines whether the detected moving object regarded as an occupant is an adult or a child, based on the occupant space distribution contained in the vehicle interior three-dimensional image. Any method may be used for the determination based on the size of the occupant space distribution, but for example, a mode may be considered in which it is determined whether the moving object regarded as an occupant is an adult or an infant from the size of the occupant space distribution. Also, for example, a mode may be considered in which it is determined whether the moving object regarded as an occupant is an adult or an infant by using a model that has learned the size of the occupant space distribution for adults and infants. When there are multiple moving objects that are regarded as occupants, the first interior determination unit 132 determines whether each of the moving objects is an adult or a child.
  • the first vehicle interior determination unit 132 determines whether or not a moving object determined to be a child is present among the moving objects regarded as occupants. When a moving object determined to be an infant is present among the moving objects regarded as occupants, the first vehicle interior determination unit 132 determines whether or not a moving object determined to be an adult is present in addition to the moving object determined to be an infant. When a moving object determined to be an adult is not present in addition to the moving object determined to be an infant, the first vehicle interior determination unit 132 determines that a moving object regarded as an infant is present in the vehicle interior.
  • the first interior determination unit 132 determines that there are no moving objects considered to be infants in the vehicle cabin.
  • the first vehicle interior determination unit 132 outputs the determination result as to whether or not a moving object regarded as a small child is present in the vehicle interior (hereinafter referred to as the “first vehicle interior determination result”) to the abandonment determination device 140 .
  • the first vehicle interior determination result includes information indicating whether or not there is a moving object that is regarded as a small child inside the vehicle interior.
  • the second vehicle interior determination unit 133 determines whether or not a moving object regarded as an infant is present in the vehicle interior (hereinafter referred to as "second vehicle interior determination") based on the biometric information acquired by the biometric information acquisition unit 1301.
  • second vehicle interior determination process the process of the second vehicle interior determination by the second vehicle interior determination unit 133 is also referred to as “second vehicle interior determination process.” The second vehicle interior determination process performed by the second vehicle interior determination unit 133 will now be described in detail.
  • the second vehicle interior determination unit 133 determines from the biometric information, in other words, from the extracted Doppler component, whether or not the characteristics of the Doppler component are associated with minute movements such as human breathing.
  • the second vehicle interior determination unit 133 determines that a moving object that is regarded as an infant is present in the vehicle interior. Note that it is not possible to distinguish between adults and infants from the characteristics of the Doppler components.
  • the second vehicle interior determination unit 133 determines that the characteristics of the Doppler components are associated with minute movements such as human breathing, it determines that the object is due to infant breathing.
  • the second interior determination unit 133 determines that there is no moving object that can be regarded as a child in the vehicle interior.
  • the second vehicle interior determination unit 133 outputs the determination result as to whether or not a moving object regarded as a small child is present in the vehicle interior (hereinafter referred to as the “second vehicle interior determination result”) to the abandonment determination device 140 .
  • the second vehicle interior determination result includes information indicating whether or not there is a moving object that is deemed to be a small child inside the vehicle interior.
  • the first vehicle interior judgment and the second vehicle interior judgment are collectively referred to simply as the “vehicle interior judgment.”
  • the first vehicle interior judgment result and the second vehicle interior judgment result are collectively referred to simply as the “vehicle interior judgment result.”
  • the outside-vehicle determination unit 135 determines whether or not there is a moving object that is regarded as a person outside the vehicle cabin (hereinafter referred to as "outside-vehicle determination") based on the three-dimensional image acquired by the three-dimensional image acquisition unit 1302.
  • the process of the outside-vehicle determination performed by the outside-vehicle determination unit 135 is also referred to as “outside-vehicle determination process.”
  • the outside-of-vehicle determination process performed by the outside-of-vehicle determination unit 135 will now be described in detail.
  • the outside-of-vehicle determination unit 135 determines whether or not a moving object that is regarded as a person is present in the portion of the three-dimensional image that corresponds to the area outside the vehicle cabin.
  • each grid of the three-dimensional image is associated with a coordinate in the real space inside the vehicle cabin, which is expressed in a three-dimensional coordinate system that represents the real space inside the vehicle cabin, so that the vehicle exterior determination unit 135 can identify an area of the three-dimensional image that corresponds to an area outside the vehicle cabin.
  • the three-dimensional image of the portion that corresponds to the area outside the vehicle cabin will be referred to as an "outside vehicle cabin three-dimensional image.”
  • the outside-compartment determination unit 135 determines, based on the shape of the spatial distribution included in the outside-compartment three-dimensional image, whether or not the spatial distribution is a spatial distribution corresponding to a person (hereinafter referred to as a "human spatial distribution"). Any method can be used for the determination based on the shape of the spatial distribution, but one possible method is to use a model that has learned the shape of the human spatial distribution to determine whether the spatial distribution contained in the three-dimensional image outside the vehicle cabin is a human spatial distribution.
  • the outside-of-vehicle determination unit 135 may determine that the spatial distribution included in the three-dimensional image of the outside of the vehicle is a human spatial distribution if the relative speed vn corresponding to the spatial distribution included in the three-dimensional image of the outside of the vehicle is greater than a preset threshold value (hereinafter referred to as a "second speed determination threshold value").
  • the second speed determination threshold value is stored in a location that the outside-of-vehicle determination unit 135 can refer to.
  • the outside-of-vehicle determination unit 135 determines that the spatial distribution contained in the outside-vehicle three-dimensional image is a human spatial distribution, it determines that a moving object regarded as a person is present outside the vehicle cabin. On the other hand, if the outside-compartment determination unit 135 determines that the spatial distribution contained in the outside-compartment three-dimensional image is not a human spatial distribution, it determines that there is no moving object that is considered to be a person outside the vehicle compartment. Unlike the first interior determination unit 132, the exterior determination unit 135 does not determine the position or physique (adult or child) of a moving object deemed to be a person, but only determines whether or not there is a moving object deemed to be a person outside the vehicle cabin.
  • the outside vehicle compartment determination unit 135 outputs the determination result as to whether or not there is a moving object regarded as a person outside the vehicle compartment (hereinafter referred to as the “outside vehicle compartment determination result”) to the abandonment determination device 140 .
  • the outside-of-vehicle determination result includes information indicating whether or not there is a moving object that is deemed to be a person outside the vehicle cabin.
  • the abandoned vehicle determination device 140 includes an interior determination result acquisition unit 141 , an exterior determination result acquisition unit 142 , a communication status information acquisition unit 143 , and the abandoned vehicle determination unit 14 .
  • the abandonment determination unit 14 includes a first determination unit 144 and a second determination unit 145 .
  • the vehicle interior determination result acquisition unit 141 acquires the vehicle interior determination result from the vehicle interior determination unit 131 of the biometric determination device 130. More specifically, the vehicle interior determination result acquisition unit 141 acquires the first vehicle interior determination result from the first vehicle interior determination unit 132 of the biometric determination device 130, and acquires the second vehicle interior determination result from the second vehicle interior determination unit 133 of the biometric determination device 130. The vehicle interior determination result acquisition unit 141 outputs the acquired vehicle interior determination result to the abandoned vehicle determination unit 14 . In addition, the abandoned vehicle determination unit 14 may have the function of the vehicle interior determination result acquisition unit 141.
  • the outside-vehicle determination result acquisition unit 142 acquires the outside-vehicle determination result from the outside-vehicle determination unit 135 of the biometric determination device 130.
  • the outside vehicle determination result acquisition unit 142 outputs the acquired outside vehicle determination result to the abandoned vehicle determination unit 14 .
  • the abandoned vehicle determination unit 14 may have the function of the outside vehicle compartment determination result acquisition unit 142.
  • the communication state information acquisition unit 143 acquires communication state information from the wireless communication state monitoring unit 41 of the vehicle information device 40 .
  • the communication status information acquisition unit 143 outputs the acquired communication status information to the abandonment determination unit 14 .
  • the abandonment determination unit 14 may have the function of the communication status information acquisition unit 143.
  • the abandonment determination unit 14 determines whether or not a child has been left behind in the vehicle cabin based on the vehicle interior determination result of the vehicle interior determination performed by the vehicle interior determination unit 131 in the biometric determination device 130, the vehicle exterior determination result of the vehicle exterior determination performed by the vehicle exterior determination unit 135 in the biometric determination device 130, and the communication status information obtained from the wireless communication status monitoring unit 41 of the vehicle information equipment 40.
  • the abandonment determination unit 14 includes the first determination unit 144 and the second determination unit 145 .
  • the first determination unit 144 determines whether or not an infant has been left behind inside the vehicle cabin (hereinafter referred to as the "first abandonment determination") based on the vehicle cabin determination result of the vehicle cabin determination performed by the vehicle cabin determination unit 131 and the vehicle exterior determination result of the vehicle exterior determination performed by the vehicle exterior determination unit 135.
  • the process in which the first determination unit 144 performs the first abandonment determination is also referred to as the "first abandonment determination process.”
  • the determination made by the first determination unit 144 as to whether or not an infant has been left behind in the vehicle cabin is a provisional determination.
  • the actual determination, which finalizes the determination result as to whether or not an infant has been left behind in the vehicle cabin, is made by the second determination unit 145 in response to the provisional determination result made by the first determination unit 144. Details of the second determination unit 145 will be described later.
  • the first abandonment determination process performed by the first determination unit 144 will now be described in detail.
  • the first determination unit 144 determines whether or not a child has been left behind inside the vehicle cabin based on the vehicle interior determination result and the vehicle exterior determination result in a redundant manner. For example, the first judgment unit 144 judges whether or not a child has been left behind in the vehicle cabin based on a table (hereinafter referred to as the ⁇ first table for leaving behind judgment'') in which the vehicle cabin interior judgment result, the vehicle exterior judgment result, and information indicating whether or not a child has been left behind in the vehicle cabin are associated with each other.
  • the first table for abandonment determination is generated in advance by an administrator or the like, and is stored in a location that can be referenced by the first determination unit 144 .
  • FIG. 10 is a diagram showing an example of the first abandonment determination table used by the first determination unit 144 in the first abandonment determination process in the first embodiment.
  • vehicle interior detection/determination result indicates the vehicle interior detection result
  • infant present indicates that the first vehicle interior determination result is information that there is a moving object considered to be an infant in the vehicle interior
  • no infant present indicates that the first vehicle interior determination result is information that there is no moving object considered to be an infant in the vehicle interior.
  • “Infant present (using biological information)” indicates that the second vehicle interior determination result is information that there is a moving object considered to be an infant in the vehicle interior
  • “no infant present (using biological information)” indicates that the second vehicle interior determination result is information that there is no moving object considered to be an infant in the vehicle interior
  • “Vehicle exterior detection/determination result” indicates the vehicle exterior detection result. In the first table for determining whether an infant has been left behind shown in FIG. 10, the "result of primary infant leaving behind determination” indicates information indicating whether an infant has been left behind in the vehicle compartment.
  • the first determination unit 144 compares the vehicle interior determination result and the vehicle exterior determination result with a first table for determining whether or not a child has been left behind inside the vehicle, as shown in FIG. 10.
  • the first judgment unit 144 will judge that an infant has been left behind in the vehicle cabin, in other words, that an infant has been left behind in the vehicle cabin, regardless of the content of the second interior judgment result.
  • the first interior vehicle interior judgment result being a judgment result indicating the presence of a moving object deemed to be a small child in the vehicle interior means, in other words, that the first interior vehicle interior judgment unit 132 in the biometric judgment device 130 has determined in the first interior vehicle interior judgment that a moving object deemed to be a small child is present in the vehicle interior.
  • the first vehicle cabin determination result has a high degree of reliability. Therefore, if the first vehicle cabin determination determines that there is a moving object deemed to be an infant inside the vehicle cabin, the first determination unit 144 can determine that an infant has been left behind inside the vehicle cabin, regardless of the second vehicle cabin determination result and the outside vehicle cabin determination result.
  • the first determination unit 144 determines whether or not there is an infant left behind in the vehicle interior, in other words, whether or not there is an occurrence of an infant being left behind in the vehicle interior. In other words, the first determination unit 144 sets the first abandonment determination to "suspend.” In other words, the first determination unit 144 does not determine whether or not there is an infant left behind in the vehicle interior.
  • the first interior vehicle interior judgment result being a judgment result indicating that there is no moving object deemed to be a small child in the vehicle interior means, in other words, that the first interior vehicle interior judgment unit 132 in the biometric judgment device 130 has determined in the first interior vehicle interior judgment that there is no moving object deemed to be a small child in the vehicle interior.
  • the second interior judgment result being a judgment result indicating the presence of a moving object deemed to be a small child in the vehicle cabin means, in other words, that the second interior judgment unit 133 in the biometric judgment device 130 has determined, in the second interior judgment, that a moving object deemed to be a small child is present in the vehicle cabin.
  • the outside-vehicle determination result being a determination result indicating the presence of a moving object deemed to be a person outside the vehicle cabin means, in other words, that the outside-vehicle determination unit 135 in the biometric determination device 130 has determined, in the outside-vehicle determination, that a moving object deemed to be a person is outside the vehicle cabin.
  • the first determination unit 144 in the first vehicle interior determination based on the vehicle interior three-dimensional image, even if a moving object deemed to be an infant is present, it may not be possible to determine this due to the influence of a child seat or vehicle structures blocking radio waves from the radio wave sensor 11. Therefore, when it is determined in the first vehicle interior determination that there is no moving object deemed to be an infant, the first determination unit 144 utilizes the second vehicle interior determination result of the second vehicle interior determination based on biological information. As a result, if it is determined in the second interior determination that a moving object deemed to be a small child is present in the vehicle interior, the first determination unit 144 further utilizes the result of the exterior determination.
  • the person inside the vehicle cabin may be regarded as an occupant in the vehicle cabin, and it may be determined as if there is an occupant in the vehicle cabin even though there is no occupant in the vehicle cabin. Therefore, even if the second vehicle interior determination determines that a moving object regarded as an infant is present in the vehicle interior, if the vehicle exterior determination determines that a moving object regarded as a person is present outside the vehicle interior, the moving object regarded as an infant that is determined to be present in the vehicle interior may not actually be present in the vehicle interior. In other words, the second vehicle interior determination unit 133 may have made an erroneous determination in the second vehicle interior determination process.
  • the alarm control unit 150 may output a false alarm in the subsequent alarm output control in the alarm control unit 150.
  • the first judgment unit 144 regards this combination of judgment results as a combination that may be an erroneous judgment, sets a status that makes the judgment result of the first abandonment judgment as to whether or not an infant has been left inside the vehicle interior indefinite, puts the first abandonment judgment on hold, and suppresses erroneous judgment of the first abandonment judgment.
  • the first judgment unit 144 will judge that an infant has been left behind in the vehicle cabin, in other words, an infant has been left behind in the vehicle cabin.
  • the outside-vehicle determination result being a determination result indicating that there is no moving object that can be considered to be a person outside the vehicle cabin means, in other words, that the outside-vehicle determination unit 135 in the biometric determination device 130 has determined, in the outside-vehicle determination, that there is no moving object that can be considered to be a person outside the vehicle cabin.
  • the first determination unit 144 can determine that there is no case of an infant being left behind in the vehicle compartment.
  • the first judgment unit 144 will disregard the outside vehicle cabin judgment result and determine that no infant has been left behind in the vehicle cabin, in other words, that no infant has been left behind in the vehicle cabin.
  • the second interior judgment result being a judgment result indicating that there is no moving object deemed to be a small child in the vehicle cabin means, in other words, that the second interior judgment unit 133 in the biometric judgment device 130 has determined in the second interior judgment that there is no moving object deemed to be a small child in the vehicle cabin.
  • the first determination unit 144 When it is determined in the first vehicle interior determination that there is no moving object that is deemed to be a small child, the first determination unit 144 utilizes the second vehicle interior determination result of the second vehicle interior determination based on the biological information. As a result, if the second interior judgment determines that there is no moving object considered to be a small child inside the vehicle cabin, the first judgment unit 144 can determine that a small child has not been left behind inside the vehicle cabin, regardless of the result of the exterior judgment.
  • the first determination unit 144 determines whether or not an infant has been left behind inside the vehicle cabin based on, for example, the vehicle interior determination result, the vehicle exterior determination result, and the first table for determination of whether an infant has been left behind.
  • the contents of the first abandoned vehicle determination table shown in Fig. 10 are merely an example.
  • the first abandoned vehicle determination result for the combination of the inside-vehicle determination result and the outside-vehicle determination result can be appropriately set according to the intention (policy) of a designer such as an administrator.
  • the method of performing the first abandonment determination by the first determination unit 144 as described above is merely one example.
  • the first judgment unit 144 may perform a first abandonment judgment by inputting the vehicle interior judgment result output from the vehicle interior judgment result acquisition unit 141 and the vehicle exterior judgment result output from the vehicle exterior judgment result acquisition unit 142 into a trained model that receives the vehicle interior judgment result and the vehicle exterior judgment result as input and outputs information indicating whether or not an infant has been left behind in the vehicle interior, and obtaining information indicating whether or not an infant has been left behind in the vehicle interior.
  • the first determination unit 144 outputs the determination result as to whether or not an infant has been left behind in the vehicle compartment (hereinafter referred to as the “first abandonment determination result”) to the second determination unit 145 .
  • the first abandoned child determination result includes information indicating whether or not an infant has been left behind in the vehicle compartment or whether it is uncertain (pending).
  • the second determination unit 145 determines whether or not an infant has been left behind in the vehicle cabin (hereinafter referred to as the "second abandonment determination") based on the first abandonment determination result of the first abandonment determination performed by the first determination unit 144 and the communication status information acquired by the communication status information acquisition unit 143.
  • the process in which the second determination unit 145 performs the second abandonment determination is also referred to as the "second abandonment determination process.”
  • the determination made by the second determination unit 145 as to whether or not an infant has been left behind in the vehicle cabin is a main determination that finalizes the determination result as to whether or not an infant has been left behind in the vehicle cabin.
  • the second determination unit 145 determines whether or not a child has been left behind in the vehicle cabin based on the first abandonment determination result and the communication status information in a redundant manner.
  • the second judgment unit 145 judges whether or not a child has been left behind in the vehicle cabin based on a table (hereinafter referred to as the ⁇ second table for leaving behind judgment'') in which the first leaving behind judgment result, information indicating whether wireless communication with the short-range wireless communication device 45 has been established, and information indicating whether or not a child has been left behind in the vehicle cabin are associated.
  • the second table for abandonment determination is generated in advance by an administrator or the like, and is stored in a location that the second determination unit 145 can refer to.
  • FIG. 11 is a diagram showing an example of the second abandonment determination table used by the second determination unit 145 in the second abandonment determination process in the first embodiment.
  • the "primary infant leaving behind determination result” indicates the first infant leaving behind determination result
  • the "wireless communication status” indicates whether wireless communication has been established between the vehicle and the short-range wireless communication device 45
  • the "secondary infant leaving behind determination result” indicates information indicating whether an infant has been left behind in the vehicle cabin.
  • the second determination unit 145 checks the first abandonment determination result and the communication status information against a second abandonment determination table as shown in FIG. 11 to determine whether or not an infant has been left behind in the vehicle compartment.
  • the second determination unit 145 can determine whether or not wireless communication with the short-range wireless communication device 45 has been established from the communication state information.
  • the second determination unit 145 assumes, for example, that the vehicle owner is in the vicinity of the vehicle and would be able to recognize the presence of an infant inside the vehicle, and determines that no infant has been left behind inside the vehicle, in other words, that no infant has been left behind inside the vehicle.
  • the second judgment unit 145 judges that an infant has not been left behind in the vehicle cabin.
  • the first abandonment judgment result being "infant abandoned” means, in other words, that the first abandonment judgment unit 14 has judged in the first abandonment judgment that an infant has been abandoned in the vehicle compartment.
  • the first abandonment judgment result being "infant abandoned or undetermined (pending)” means, in other words, that the first abandonment judgment unit 14 has put the first abandonment judgment on hold.
  • the second judgment unit 145 judges that an infant has been left behind in the vehicle cabin.
  • the second determination unit 145 assumes, for example, that the owner of the vehicle has left the vehicle without realizing that a small child has been left behind inside the vehicle.
  • the second determination unit 145 determines whether or not a child has been left behind in the vehicle compartment based on, for example, the first abandonment determination result, the communication status information, and the second abandonment determination table.
  • the contents of the second table for determining whether a vehicle is abandoned, as shown in Fig. 11, are merely an example.
  • the second abandonment determination result for the combination of the first abandonment determination result and whether wireless communication between the vehicle and the short-range wireless communication device 45 is established or not can be set as appropriate based on the intention (policy) of a designer such as an administrator.
  • the method of performing the second abandonment determination by the second determination unit 145 as described above is merely one example.
  • the second judgment unit 145 may perform the second abandonment judgment by inputting the first abandonment judgment result output from the first judgment unit 144 and the communication status information output from the communication status information acquisition unit 143 into a trained model that receives the first abandonment judgment result and communication status information and outputs information indicating whether or not an infant has been left behind in the vehicle cabin, and obtaining information indicating whether or not an infant has been left behind in the vehicle cabin.
  • the second determination unit 145 outputs the determination result as to whether or not an infant has been left behind in the vehicle compartment (hereinafter referred to as the “second abandonment determination result”) to the alarm control device 150 .
  • the second abandoned child determination result includes information indicating whether or not an infant has been left behind in the vehicle compartment.
  • the second determination unit 145 outputs the second abandonment determination result to the alarm control device 150 together with the first abandonment determination result.
  • the first abandonment determination result and the second abandonment determination result are collectively referred to simply as the "abandonment determination result.”
  • the alarm control device 150 includes an abandonment determination result acquisition unit 151 and an alarm control unit 152 .
  • the abandonment determination result acquisition unit 151 acquires the abandonment determination result from the abandonment determination device 140 . More specifically, the abandonment determination result acquisition unit 151 acquires the first abandonment determination result and the second determination result from the abandonment determination device 140 . The abandonment determination result acquisition unit 151 outputs the acquired abandonment determination result to the alarm control unit 152.
  • the alarm control unit 152 outputs an instruction to the alarm device 160 to output an alarm (hereinafter referred to as an "alarm instruction").
  • the warning control unit 152 can select whether to output a warning instruction based on the first abandonment determination result or based on the second abandonment determination result.
  • the alarm control unit 152 can select whether to output an alarm instruction based on the result of a first abandonment judgment by the first judgment unit 144, in other words, a provisional judgment, or to output an alarm instruction based on the result of a second abandonment judgment by the second judgment unit 145, in other words, a main judgment.
  • the alarm control unit 152 determines whether or not to output an alarm instruction based on a table (hereinafter referred to as the "first alarm table”) in which the first abandonment judgment result is associated with information indicating whether an alarm should be output, or a table (hereinafter referred to as the "second alarm table”) in which the second abandonment judgment result is associated with information indicating a child for which an alarm should be output.
  • the first alarm table and the second alarm table are generated in advance by an administrator or the like, and are stored in a location that the alarm control unit 152 can refer to.
  • the warning control unit 152 uses the first warning table to determine whether or not a warning output is required.
  • the warning control unit 152 uses the second warning table to determine whether or not a warning output is required.
  • FIG. 12 is a diagram showing an example of the first alarm table and the second alarm table used by the alarm control unit 152 to determine whether to output an alarm instruction in the first embodiment.
  • FIG. 12A shows an example of a first alarm table
  • FIG. 12B shows an example of a second alarm table.
  • first table for warning shown in FIG. 12A "Yes", “Undefined”, and “No” indicate the first abandonment judgment result, and "Warning ON” and “Warning OFF” indicate whether or not a warning output is required.
  • “Yes” and “No” indicate the second abandonment determination result, and "Warning ON” and “Warning OFF” indicate whether or not a warning output is required.
  • the alarm control unit 152 selects to output an alarm instruction based on the first abandonment determination result, if the first abandonment determination result indicates that an infant is left in the vehicle compartment (pending), in other words, if the first determination unit 144 has put the first abandonment determination on hold, the alarm control unit 152 determines that an alarm output is unnecessary (alarm OFF). In this case, the alarm control unit 152 does not output an alarm instruction.
  • the first determination unit 144 reserves the first abandonment determination when there is a possibility that the first abandonment determination is an erroneous determination.
  • the alarm control unit 152 determines that an alarm output is unnecessary in order to eliminate the inconvenience of an erroneous determination of the first abandonment determination, that is, a false alarm due to an erroneous determination of whether or not an infant has been left behind in the vehicle compartment.
  • the alarm control unit 152 selects to output an alarm instruction based on the second abandonment determination result, if the second abandonment determination result indicates that an infant has been left behind in the vehicle interior, in other words, if the second determination unit 145 determines that an infant has been left behind in the vehicle interior, the alarm control unit 152 determines that an alarm output is necessary (alarm ON). In this case, the alarm control unit 152 outputs an alarm instruction.
  • the alarm control unit 152 may output an alarm instruction when a state in which it has been determined that an alarm output is necessary continues for a preset time period (hereinafter referred to as the "alarm determination time period") or longer.
  • the alarm determination time period is, for example, 10 seconds. Note that this is merely an example, and the alarm determination time period can be any length as appropriate.
  • the alarm device 160 outputs an alarm based on the control of the alarm control device 150 .
  • the alarm device 160 outputs an alarm when an alarm instruction is output from the alarm control device 150 .
  • the warning device 160 displays a message to the effect that a vehicle has been abandoned inside the vehicle cabin.
  • the warning device 160 may output a buzzer, or may output a voice message to the effect that a vehicle has been abandoned inside the vehicle cabin.
  • the warning device 160 is a horn provided in a vehicle, the warning device 160 will sound.
  • the warning device 160 is a light provided in a vehicle, the warning device 160 will flash.
  • the first determination unit 144 in the abandonment determination device 140 reserves the first abandonment determination when there is a possibility that the first abandonment determination is an erroneous determination, thereby suppressing erroneous determination that an infant has been abandoned in the vehicle cabin, and the alarm control unit 152 in the alarm control device 150 does not output an alarm instruction based on the first abandonment determination result indicating that the first abandonment determination has been reserved, thereby suppressing false alarms that an infant has been abandoned in the vehicle cabin.
  • the abandonment determination system 100 can solve the above practical problems.
  • the second determination unit 145 in the abandonment determination device 140 performs the second abandonment determination by combining the three-dimensional image and biometric information output from the radio wave sensor 11 with the communication status information, so that the second determination unit 145 can suppress erroneous determination that an infant has been abandoned in the vehicle cabin and can prevent unnecessary alarms to the vehicle owner, etc., which can be considered as false alarms.
  • the alarm control unit 152 in the alarm control device 150 controls the output of the alarm instruction based on the second abandonment determination result that takes into account the communication status between the vehicle and the short-range wireless communication device 45, so that it can suppress false alarms that an infant has been abandoned in the vehicle cabin and can suppress unnecessary alarms to the vehicle owner, etc., which can be considered as false alarms. In this way, the abandonment determination system 100 can suppress false alarms that an infant has been abandoned in the vehicle cabin and can suppress unnecessary alarms to the vehicle owner, etc.
  • FIG. 13 is a flowchart for explaining the operation of the abandonment determination system 100 according to the first embodiment.
  • the abandoned vehicle determination system 100 repeats the operation shown in the flowchart of FIG. 13, for example, when the vehicle doors are locked, the vehicle power is turned off, or the shift position in the vehicle is set to "P," until the vehicle doors are unlocked or the vehicle power is turned on.
  • the processes of step ST1-1, and steps ST1-2 to ST3 are performed by the live body determination device 130, and the processes of step ST4-1, step ST4-2, and steps ST4-3 to ST6 are performed by the abandonment determination device 140.
  • the biometric information acquisition unit 1301 acquires the biometric information extracted by the biometric information extraction unit 122 of the sensor signal processing unit 120 (step ST1-1).
  • the biometric information acquisition unit 1301 outputs the acquired biometric information to the second vehicle interior determination unit 133 of the vehicle interior determination unit 131.
  • the three-dimensional image acquisition unit 1302 acquires the three-dimensional image generated by the three-dimensional image generation unit 121 of the sensor signal processing unit 120 (step ST1-2).
  • the three-dimensional image acquisition unit 1302 outputs the acquired three-dimensional image to the first vehicle interior determination unit 132 of the vehicle interior determination unit 131 and the vehicle exterior determination unit 135 .
  • the vehicle interior determination unit 131 executes a vehicle interior determination process to determine whether or not a moving object deemed to be a child is present in the vehicle interior based on the three-dimensional image acquired by the three-dimensional image acquisition unit 1302 in step ST1-2 and the biometric information acquired by the biometric information acquisition unit 1301 in step ST1-1 (step ST2).
  • the vehicle interior determination unit 131 outputs the vehicle interior determination result to the abandoned vehicle determination device 140 .
  • the outside-of-vehicle determination unit 135 executes an outside-of-vehicle determination process based on the three-dimensional image acquired by the three-dimensional image acquisition unit 1302 in step ST1-2 (step ST3).
  • the outside-vehicle determination unit 135 outputs the outside-vehicle determination result to the abandoned vehicle determination device 140 .
  • the vehicle interior determination result acquisition unit 141 acquires the vehicle interior determination result output from the vehicle interior determination unit 131 (step ST4-1).
  • the vehicle interior determination result acquisition unit 141 outputs the acquired vehicle interior determination result to the abandoned vehicle determination unit 14 .
  • the outside-vehicle determination result acquisition unit 142 acquires the outside-vehicle determination result from the outside-vehicle determination unit 135 (step ST4-2).
  • the outside vehicle determination result acquisition unit 142 outputs the acquired outside vehicle determination result to the abandoned vehicle determination unit 14 .
  • the communication state information acquisition unit 143 acquires communication state information from the wireless communication state monitoring unit 41 of the vehicle information device 40 (step ST4-3).
  • the communication status information acquisition unit 143 outputs the acquired communication status information to the abandonment determination unit 14 .
  • the first judgment unit 144 of the abandonment judgment unit 14 performs a first abandonment judgment process to make a first abandonment judgment based on the vehicle interior judgment result acquired by the vehicle interior judgment result acquisition unit 141 in step ST4-1 and the vehicle exterior judgment result acquired by the vehicle exterior judgment unit in step ST4-2 (step ST5).
  • the first determination unit 144 outputs the first abandonment determination result to the second determination unit 145 .
  • the second judgment unit 145 of the abandonment judgment unit 14 performs a second abandonment judgment process to make a second abandonment judgment based on the first abandonment judgment result of the first abandonment judgment performed by the first judgment unit 144 in step ST5 and the communication status information acquired by the communication status information acquisition unit 143 in step ST4-3 (step ST6).
  • the second determination unit 145 outputs the abandonment determination result to the alarm control device 150.
  • the second determination unit 145 outputs the second abandonment determination result to the alarm control device 150 together with the first abandonment determination result.
  • step ST1-1 and the process of step ST1-2 are performed in parallel, but this is merely an example.
  • the order of the process of step ST1-1 and the process of step ST1-2 may be reversed.
  • step ST4-1, the process of step ST4-2, and the process of step ST4-3 are performed in parallel, but this is merely an example.
  • the order of the process of step ST4-1, the process of step ST4-2, and the process of step ST4-3 may be changed.
  • FIG. 14 is a flowchart for explaining in detail the process of step ST2 in FIG.
  • the first vehicle interior determination unit 132 executes a first vehicle interior determination process to make a first vehicle interior determination as to whether or not a moving object deemed to be a child is present in the vehicle interior based on the three-dimensional image acquired by the three-dimensional image acquisition unit 1302 in step ST1-2 of Figure 13 (step ST1401).
  • the first vehicle interior determination unit 132 outputs the first vehicle interior determination result to the abandoned vehicle determination device 140.
  • the second vehicle interior determination unit 133 executes a second vehicle interior determination process to make a second vehicle interior determination as to whether or not a moving object deemed to be a child is present in the vehicle interior based on the biometric information acquired by the biometric information acquisition unit 1301 in step ST1-1 of Figure 13 (step ST1402).
  • the second vehicle interior determination unit 133 outputs the second vehicle interior determination result to the abandoned vehicle determination device 140.
  • step ST1401 is performed first, followed by the processing of step ST1402, but this is merely an example.
  • the processing of step ST1401 and the processing of step ST1402 may be performed in parallel, or the order of the processing of step ST1401 and the processing of step ST1402 may be reversed.
  • the abandonment determination system 100 performs an inside-vehicle determination as to whether or not a moving object regarded as an infant is present inside the vehicle, based on the three-dimensional image generated by the radio wave sensor 11 and the biological information extracted by the radio wave sensor 11.
  • the abandonment determination system 100 also performs an outside-vehicle determination as to whether or not a moving object regarded as a person is present outside the vehicle, based on the three-dimensional image.
  • the abandonment determination system 100 determines whether or not an infant has been left behind inside the vehicle, based on the result of the inside-vehicle determination (inside-vehicle determination result) and the result of the outside-vehicle determination (outside-vehicle determination result). Therefore, the abandonment determination system 100 can prevent erroneous determination of whether or not a small child has been left behind in the vehicle.
  • FIG. 15 is a flowchart for explaining the operation of the alarm control device 150 according to the first embodiment.
  • the alarm control device 150 carries out an operation as shown in the flowchart of FIG.
  • the abandonment determination result acquisition unit 151 acquires the abandonment determination result from the abandonment determination device 140 (step ST101). More specifically, the abandonment determination result acquisition unit 151 acquires the first abandonment determination result and the second determination result from the abandonment determination device 140 . The abandonment determination result acquisition unit 151 outputs the acquired abandonment determination result to the alarm control unit 152.
  • the alarm control unit 152 outputs an alarm instruction to the alarm device 160 when the first determination unit 144 of the abandonment determination device 140 provisionally determines that an infant has been left behind in the vehicle cabin based on the abandonment determination result acquired by the abandonment determination result acquisition unit 151 in step ST101, or when the second determination unit 145 of the abandonment determination device 140 determines that an infant has been left behind in the vehicle cabin.
  • the alarm control device 150 outputs an alarm instruction to output an alarm when the abandonment determination device 140 determines that an infant has been left behind inside the vehicle. This allows the alarm control device 150 to inform the vehicle owner or people in the vicinity of the vehicle that an infant has been left behind inside the vehicle.
  • the alarm control device 150 does not output an alarm instruction when the abandonment determination device 140 reserves the determination as to whether or not a child has been left behind in the vehicle compartment. Therefore, the alarm control device 150 can suppress false alarms when, for example, it is erroneously determined that an infant has been left behind in the vehicle compartment when in fact the infant has not been left behind.
  • the alarm control device 150 can suppress false alarms that a child has been left behind in the vehicle cabin by controlling the output of an alarm instruction based on the result of the determination (second abandonment determination result) made by the abandonment determination device 140 of whether or not a child has been left behind in the vehicle cabin by combining the three-dimensional image and biometric information output from the radio wave sensor 11 with the communication status information. For example, even if only a small child is present in the vehicle, if the vehicle owner is outside the vehicle and wireless communication is established between the vehicle and the short-range wireless communication device 45 carried by the vehicle owner, it is assumed that the vehicle owner is aware of the small child in the vehicle. In this case, the alarm is unnecessary, and if an alarm is output, it can be said that the output alarm is a false alarm. The alarm control device 150 can suppress such false alarms.
  • the first judgment unit 144 of the abandonment judgment device 140 judges to put on hold the judgment of whether abandonment has occurred (first abandonment judgment)
  • the put on hold state continues for a predetermined time (hereinafter referred to as the "first pending judgment time") or more
  • the first pending judgment time a predetermined time
  • the owner of the vehicle is outside the vehicle cabin and at a distance where the short-range wireless communication device 45 carried by the owner can establish communication with the vehicle, it is possible that the child may enter the vehicle cabin for some reason without the owner's knowledge and be unable to get out.
  • the first suspension determination time may be determined to be a time that an infant has been left behind in the vehicle cabin.
  • the first suspension determination time is, for example, 10 minutes. Note that this is merely an example, and the first suspension determination time may be longer than the warning determination time.
  • the first judgment unit 144 may notify the second judgment unit 145 that it has changed the first abandonment judgment result from a pending state to a judgment that an infant has been left behind, and the second judgment unit 145 may determine that an infant has been left behind in the vehicle cabin regardless of the content of the communication status information.
  • the abandonment determination system 100 reserves the right to suspend a determination that a child has been left behind in a vehicle cabin if there is a possibility that the determination is incorrect, and by preventing an alarm based on an incorrect determination from being output, it is possible to prevent a situation in which a truly necessary alarm is not output.
  • the warning control unit 152 of the warning control device 150 may be configured to output a warning instruction when the state in which the first judgment unit 144 of the abandonment judgment device 140 has reserved the judgment of whether abandonment has occurred (first abandonment judgment) continues for a preset time or more (hereinafter referred to as the "second pending judgment time").
  • the second pending judgment time is, for example, 10 minutes. Note that this is merely an example, and the second pending judgment time may be any time longer than the warning judgment time.
  • the alarm control device 150 can prevent situations in which a truly necessary alarm is not output, such as when, for example, a small child enters the vehicle cabin for some reason without the owner's knowledge and is unable to get out, even if the owner is outside the vehicle cabin and is close enough that the short-range wireless communication device 45 carried by the owner can establish communication with the vehicle.
  • the alarm control unit 152 is capable of selecting whether to output an alarm instruction based on the first abandonment judgment result or to output an alarm instruction based on the second abandonment judgment result, but this is merely one example.
  • the warning control unit 152 may always output a warning instruction based on the second abandonment determination result, in which case the first warning table does not need to be provided.
  • the alarm control unit 152 of the alarm control device 150 may perform control to change the alarm means of the alarm output from the alarm device 160, for example, with the elapsed time since it was determined that an infant was left behind in the vehicle cabin.
  • the alarm control unit 152 controls the alarm device 160 to output a louder sound as the elapsed time since it was determined that an infant was left behind in the vehicle cabin increases.
  • the alarm control unit 152 may control to change the content of the alarm output from the alarm device 160, such as changing from sending a message to outputting an alarm sound, with the elapsed time since it was determined that an infant was left behind in the vehicle cabin increasing.
  • the first alarm table or the second alarm table incorporates information regarding the alarm means by which the alarm device 160 should output an alarm.
  • the abandonment determination device 140 includes the first determination unit 144 and the second determination unit 145, but this is merely one example.
  • the abandonment determination device 140 does not necessarily have to include the second determination unit 145 , and may be configured without including the second determination unit 145 .
  • the first abandonment determination made by the first determination unit 144 in the abandonment determination device 140 as to whether or not an infant has been left behind in the vehicle compartment becomes a main determination.
  • the alarm control unit 152 in the alarm control device 150 uses the first alarm table to determine whether or not an alarm output is required.
  • the first alarm table does not need to be provided.
  • the process of step ST6 in the operation of the abandonment determination system 100 described using the flowchart of FIG. 13 can be omitted.
  • the determination made by the first vehicle interior determination unit 132 in its vehicle interior determination of "whether or not there is a moving object regarded as a small child in the vehicle interior" is taken to mean “whether or not only moving objects regarded as small children are present in the vehicle interior," but this is merely one example.
  • the determination made by the first interior compartment determination unit 132 in the first interior compartment determination "whether or not there is a moving object regarded as a small child in the vehicle cabin" may mean "whether or not there is a moving object regarded as a small child in the vehicle cabin, regardless of whether or not there are other adult occupants.”
  • the first vehicle interior judgment unit 132 in the vehicle interior judgment process, outputs information that matches all moving objects considered to be occupants present in the vehicle interior with information indicating whether the moving objects are adults or children to the abandoned vehicle judgment device 140 as the first vehicle interior judgment result.
  • the first judgment unit 144 of the abandonment judgment device 140 judges whether or not there are only moving objects regarded as small children in the vehicle cabin based on the first vehicle cabin judgment result, and uses this judgment result as the first vehicle cabin judgment result for matching with the first table for abandonment judgment.
  • the radio wave sensor 11 may generate a three-dimensional image of the inside of the vehicle compartment and a three-dimensional image of the outside of the vehicle compartment separately.
  • the first vehicle interior determination unit 132 performs a first vehicle interior determination process based on the vehicle interior three-dimensional image
  • the vehicle exterior determination unit 135 performs a vehicle exterior determination process based on the vehicle exterior three-dimensional image.
  • the biometric determination device 130 is mounted on the radio wave sensor 11, but this is merely an example.
  • the biometric determination device 130 may be provided outside the radio wave sensor 11 so as to be connectable to the radio wave sensor 11.
  • the biometric determination device 130 may be mounted on the abandonment determination device 140 .
  • the abandonment determination device 140 and the alarm control device 150 may be mounted on the radio wave sensor 11 .
  • the sensor signal processing unit 120 may be provided outside the radio wave sensor 11 . The sensor signal processing unit 120, the biometric determination device 130, the abandonment determination device 140, and the alarm control device 150 can be appropriately combined into a single device.
  • the respective components provided in the sensor signal processing unit 120 biometric information extraction unit 122 and three-dimensional image generation unit 121
  • the respective components provided in the biometric determination device 130 biometric information acquisition unit 1301, three-dimensional image acquisition unit 1302, vehicle interior determination unit 131, and vehicle exterior determination unit 135)
  • the respective components provided in the abandonment determination device 140 vehicle interior determination result acquisition unit 141, vehicle exterior determination result acquisition unit 142, communication status information acquisition unit 143, and abandonment determination unit 14
  • the respective components provided in the alarm control device 150 can be appropriately combined to be provided in a single device.
  • the outside vehicle compartment determination unit 135 may be provided in the abandonment determination device 140
  • the warning control unit 152 may be provided in the abandonment determination device 140.
  • the abandonment determination unit 14 may be provided in the living body determination device 130.
  • the abandonment determination system 100 is configured by the live body determination device 130 and the abandonment determination device 140, but this is merely one example.
  • the alarm control device 150 may be provided in the abandonment determination system 100, or the sensor signal processing unit 120 may be provided in the abandonment determination system 100.
  • the living body determination device 130, the abandonment determination device 140, and the alarm control device 150 are in-vehicle devices mounted in a vehicle, but this is merely one example.
  • the live body determination device 130, the abandonment determination device 140, or the alarm control device 150 may be provided in a server (not shown).
  • biometric information acquisition unit 1301, the three-dimensional image acquisition unit 1302, the vehicle interior determination unit 131, or the vehicle exterior determination unit 135 may be provided on the server, some or all of the vehicle interior determination result acquisition unit 141, the vehicle exterior determination result acquisition unit 142, the communication status information acquisition unit 143, or the abandonment determination unit 14 may be provided on the server, or some or all of the abandonment determination result acquisition unit 151 or the alarm control unit 152 may be provided on the server.
  • FIG. 16A and 16B are diagrams illustrating an example of a hardware configuration of the abandonment determination device 140 according to the first embodiment.
  • the functions of the vehicle interior determination result acquisition unit 141, the vehicle exterior determination result acquisition unit 142, the communication state information acquisition unit 143, and the abandonment determination unit 14 are realized by the processing circuit 1001. That is, the abandonment determination device 140 includes the processing circuit 1001 for controlling the determination of whether or not an infant is left behind in the vehicle interior.
  • the processing circuitry 1001 may be dedicated hardware as shown in FIG. 16A, or may be a processor 1004 that executes a program stored in memory as shown in FIG. 16B.
  • the processing circuit 1001 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a combination of these.
  • ASIC Application Specific Integrated Circuit
  • FPGA Field-Programmable Gate Array
  • the functions of the vehicle interior determination result acquisition unit 141, the vehicle exterior determination result acquisition unit 142, the communication status information acquisition unit 143, and the abandoned vehicle determination unit 14 are realized by software, firmware, or a combination of software and firmware.
  • the software or firmware is written as a program and stored in the memory 1005.
  • the processor 1004 executes the functions of the vehicle interior determination result acquisition unit 141, the vehicle exterior determination result acquisition unit 142, the communication status information acquisition unit 143, and the abandoned vehicle determination unit 14 by reading and executing the program stored in the memory 1005.
  • the abandoned vehicle determination device 140 includes a memory 1005 for storing a program that, when executed by the processor 1004, results in the execution of steps ST4-1, ST4-2, and steps ST4-3 to ST6 in FIG. 13 described above.
  • the programs stored in the memory 1005 can be said to cause the computer to execute the procedures or methods of the vehicle interior determination result acquisition unit 141, the vehicle exterior determination result acquisition unit 142, the communication status information acquisition unit 143, and the abandoned vehicle determination unit 14.
  • the memory 1005 corresponds to, for example, a non-volatile or volatile semiconductor memory such as a RAM, a ROM (Read Only Memory), a flash memory, an EPROM (Erasable Programmable Read Only Memory), or an EEPROM (Electrically Erasable Programmable Read-Only Memory), a magnetic disk, a flexible disk, an optical disk, a compact disk, a mini disk, a DVD (Digital Versatile Disc), etc.
  • a non-volatile or volatile semiconductor memory such as a RAM, a ROM (Read Only Memory), a flash memory, an EPROM (Erasable Programmable Read Only Memory), or an EEPROM (Electrically Erasable Programmable Read-Only Memory), a magnetic disk, a flexible disk, an optical disk, a compact disk, a mini disk, a DVD (Digital Versatile Disc), etc.
  • the functions of the vehicle interior determination result acquisition unit 141, the vehicle exterior determination result acquisition unit 142, the communication status information acquisition unit 143, and the abandoned vehicle determination unit 14 may be partially realized by dedicated hardware and partially realized by software or firmware.
  • the functions of the vehicle interior determination result acquisition unit 141, the vehicle exterior determination result acquisition unit 142, and the communication status information acquisition unit 143 can be realized by the processing circuit 1001 as dedicated hardware, and the functions of the abandoned vehicle determination unit 14 can be realized by the processor 1004 reading and executing a program stored in the memory 1005.
  • the abandoned vehicle determination device 140 also includes an input interface device 1002 and an output interface device 1003 that perform wired or wireless communication with devices such as the radio wave sensor 11, the alarm control device 150, or the vehicle information device 40.
  • the hardware configuration of the biometric determination device 130 according to the first embodiment is similar to the hardware configuration of the abandonment determination device 140 described using FIGS. 16A and 16B, and therefore is not illustrated.
  • the functions of the biometric information acquisition unit 1301, the three-dimensional image acquisition unit 1302, the vehicle interior determination unit 131, and the vehicle exterior determination unit 135 are realized by the processing circuit 1001. That is, the biometric determination device 130 includes the processing circuit 1001 for performing vehicle interior determination processing and vehicle exterior determination processing based on the three-dimensional image generated by the radio wave sensor 11 and the biometric information extracted by the radio wave sensor 11.
  • the processing circuitry 1001 may be dedicated hardware as shown in FIG. 16A, or may be a processor 1004 executing a program stored in a memory 1005 as shown in FIG. 16B.
  • the processing circuit 1001 executes the functions of the biometric information acquisition unit 1301, the three-dimensional image acquisition unit 1302, the vehicle interior determination unit 131, and the vehicle exterior determination unit 135 by reading and executing a program stored in the memory 1005.
  • the biometric determination device 130 includes a memory 1005 for storing a program that, when executed by the processing circuit 1001, results in the execution of steps ST1-1 and ST1-2 to ST3 in FIG. 13 described above. It can also be said that the program stored in the memory 1005 causes a computer to execute the processing procedures or methods of the biometric information acquisition unit 1301, the three-dimensional image acquisition unit 1302, the vehicle interior determination unit 131, and the vehicle exterior determination unit 135.
  • the live body determination device 130 includes an abandoned body determination device 140 and other devices, an input interface device 1002 and an output interface device 1003 that perform wired or wireless communication with the abandoned body determination device 140 and other devices.
  • the hardware configuration of the alarm control device 150 is similar to the hardware configuration of the abandonment determination device 140 described using Figures 16A and 16B, and therefore is not illustrated.
  • the functions of the abandonment determination result acquisition unit 151 and the alarm control unit 152 are realized by the processing circuit 1001. That is, the alarm control device 150 includes the processing circuit 1001 for controlling the output of an alarm instruction based on the abandonment determination result by the abandonment determination device 140.
  • the processing circuitry 1001 may be dedicated hardware as shown in FIG. 16A, or may be a processor 1004 executing a program stored in a memory 1005 as shown in FIG. 16B.
  • the processing circuit 1001 executes the functions of the abandoned object determination result acquisition unit 151 and the alarm control unit 152 by reading and executing a program stored in the memory 1005. That is, the alarm control device 150 includes a memory 1005 for storing a program which, when executed by the processing circuit 1001, results in the execution of steps ST101 to ST102 in Fig. 15 described above. It can also be said that the program stored in the memory 1005 causes a computer to execute the processing procedures or methods of the abandoned object determination result acquisition unit 151 and the alarm control unit 152.
  • the alarm control device 150 includes an input interface device 1002 and an output interface device 1003 that perform wired or wireless communication with devices such as the abandoned object determination device 140 or the alarm device 160 .
  • the abandoned vehicle determination system 100 includes a radio wave sensor 11 that is provided in a vehicle interior and detects a moving object based on a radio wave radiated toward the vehicle interior and reflected by an object in the vehicle interior, a biometric information acquisition unit 1301 that acquires biometric information extracted based on a detection result of the moving object detected by the radio wave sensor 11, a three-dimensional image acquisition unit 1302 that acquires a three-dimensional image that three-dimensionally represents a distribution of an area in which the moving object exists in the vehicle interior, the three-dimensional image acquired by the three-dimensional image acquisition unit 1302, and
  • the vehicle is equipped with an interior determination unit 131 which performs an interior determination as to whether or not a moving object deemed to be a person requiring assistance (infant) is present inside the vehicle cabin based on the biometric information acquired by the vehicle interior determination unit 1301, an exterior determination unit 135 which performs an exterior determination as to whether or not a moving object deemed to be a person is present outside the vehicle cabin based
  • the abandonment determination unit 14 is provided with a first determination unit that, when the first vehicle interior determination unit 132 determines that there is no moving object regarded as a person requiring assistance (infant) inside the vehicle interior in the first vehicle interior determination and the second vehicle interior determination unit 133 determines that there is a moving object regarded as a person requiring assistance (infant) inside the vehicle interior in the second vehicle interior determination, determines that abandonment has occurred if the vehicle exterior determination unit 135 determines that there is no moving object regarded as a person outside the vehicle interior in the vehicle exterior determination, and reserves the determination as to whether abandonment has occurred or not if the vehicle exterior determination unit 135 determines that there is a moving object regarded as a person outside the vehicle interior in the vehicle exterior determination. Therefore, the abandonment determination system 100 can suppress erroneous determination of whether or not a person requiring assistance has been left behind in the vehicle cabin.
  • the abandonment determination unit 14 is provided with a second determination unit 145 which makes a provisional determination based on the determination by the first determination unit 144 of whether or not abandonment has occurred (first abandonment determination), and makes a final determination of whether or not abandonment has occurred (second abandonment determination) based on the result of the provisional determination and communication status information regarding the communication status with the short-range wireless communication device 45 outside the vehicle cabin, obtained from a vehicle information device 40 which is mounted inside the vehicle cabin and monitors the communication status with the short-range wireless communication device 45 outside the vehicle cabin.
  • the abandonment determination system 100 combines the three-dimensional images and biometric information output from the radio wave sensor 11 with communication status information to determine whether or not a person requiring assistance has been left behind in the vehicle cabin, thereby confirming whether or not a person requiring assistance has been left behind in the vehicle cabin and suppressing erroneous determinations that a person requiring assistance has been left behind in the vehicle cabin.
  • the first vehicle interior determination unit 132 determines that there is no moving object regarded as a person requiring assistance (infant) inside the vehicle interior in the first vehicle interior determination and the second vehicle interior determination unit 133 determines that there is a moving object regarded as a person requiring assistance inside the vehicle interior in the second vehicle interior determination
  • the vehicle exterior determination unit 135 determines that there is no moving object regarded as a person outside the vehicle interior in the vehicle exterior determination
  • the first determination unit 144 provisionally determines that abandonment has occurred
  • the second judgment unit 145 reserves the judgment as to whether or not an abandonment has occurred
  • the first judgment unit 144 provisionally judges that an abandonment has occurred or reserves the judgment as to whether or not an abandonment has occurred, the second judgment unit 145 determines that an abandonment has not occurred if communication between the vehicle and the short-range wireless communication device 45 is established based on
  • the abandonment determination system 100 combines the three-dimensional images and biometric information output from the radio wave sensor 11 with communication status information to determine whether or not a person requiring assistance has been left behind in the vehicle cabin, thereby confirming whether or not a person requiring assistance has been left behind in the vehicle cabin and suppressing erroneous determinations that a person requiring assistance has been left behind in the vehicle cabin.
  • the abandonment determination system 100 includes an alarm control unit 152 that outputs an alarm instruction to output an alarm when the first determination unit 144 determines that abandonment has occurred, and the alarm control unit 152 can be configured not to output an alarm instruction when the first determination unit 144 reserves the determination of whether abandonment has occurred. Therefore, the abandonment determination system 100 can suppress false alarms, for example, when it is erroneously determined that a person requiring assistance has been left behind in the vehicle compartment when in fact the person requiring assistance has not been left behind.
  • the abandonment determination device 140 is configured to include: a radio wave sensor 11 that is provided inside the vehicle cabin and detects moving objects based on reflected waves emitted toward the vehicle cabin and reflected by objects inside the vehicle cabin; an interior determination result acquisition unit 141 that acquires an interior determination result of whether or not a moving object deemed to be a person requiring assistance (infant) is present inside the vehicle cabin, the image being generated based on the detection result of the moving object detected by the radio wave sensor 11, and based on biometric information extracted based on the detection result of the moving object detected by the radio wave sensor 11; an exterior determination result acquisition unit 142 that acquires an exterior determination result of whether or not a moving object deemed to be a person is present outside the vehicle cabin, based on the three-dimensional image; and an abandonment determination unit 14 that determines whether or not a person requiring assistance has been left behind inside the vehicle cabin, based on the interior determination result acquired by the interior determination result acquisition unit 141 and the exterior determination result acquired by the exterior determination result acquisition unit
  • any of the components of the embodiments may be modified or omitted.
  • the abandoned vehicle detection system disclosed herein can prevent erroneous determination of whether or not a small child or other person has been left behind in the vehicle.

Landscapes

  • Engineering & Computer Science (AREA)
  • Remote Sensing (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Business, Economics & Management (AREA)
  • Emergency Management (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Electromagnetism (AREA)
  • Traffic Control Systems (AREA)
PCT/JP2022/044646 2022-12-05 2022-12-05 置き去り判定システム、置き去り判定装置、および、置き去り判定方法 Ceased WO2024121876A1 (ja)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP2024562388A JP7778250B2 (ja) 2022-12-05 2022-12-05 置き去り判定システム、置き去り判定装置、および、置き去り判定方法
PCT/JP2022/044646 WO2024121876A1 (ja) 2022-12-05 2022-12-05 置き去り判定システム、置き去り判定装置、および、置き去り判定方法

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2022/044646 WO2024121876A1 (ja) 2022-12-05 2022-12-05 置き去り判定システム、置き去り判定装置、および、置き去り判定方法

Publications (1)

Publication Number Publication Date
WO2024121876A1 true WO2024121876A1 (ja) 2024-06-13

Family

ID=91378689

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2022/044646 Ceased WO2024121876A1 (ja) 2022-12-05 2022-12-05 置き去り判定システム、置き去り判定装置、および、置き去り判定方法

Country Status (2)

Country Link
JP (1) JP7778250B2 (https=)
WO (1) WO2024121876A1 (https=)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003066157A (ja) * 2001-08-24 2003-03-05 Fujitsu Ten Ltd 車両内人体検出方法
JP2017007453A (ja) * 2015-06-19 2017-01-12 株式会社東海理化電機製作所 動体検知警告装置
US20200050874A1 (en) * 2016-09-23 2020-02-13 Iee International Electronics & Engineering S.A. Car interior surveillance system with e-call functionality
JP2020101415A (ja) * 2018-12-20 2020-07-02 パナソニックＩｐマネジメント株式会社 乗員状態検知システム
JP2020142718A (ja) * 2019-03-07 2020-09-10 日本電産モビリティ株式会社 車両制御装置
WO2021122603A1 (en) * 2019-12-20 2021-06-24 Iee International Electronics & Engineering S.A. Method for seat occupancy detection
WO2021240777A1 (ja) * 2020-05-29 2021-12-02 三菱電機株式会社 乗員検出装置および乗員検出方法

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2020134184A (ja) * 2019-02-14 2020-08-31 オムロン株式会社 乗員検出装置

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003066157A (ja) * 2001-08-24 2003-03-05 Fujitsu Ten Ltd 車両内人体検出方法
JP2017007453A (ja) * 2015-06-19 2017-01-12 株式会社東海理化電機製作所 動体検知警告装置
US20200050874A1 (en) * 2016-09-23 2020-02-13 Iee International Electronics & Engineering S.A. Car interior surveillance system with e-call functionality
JP2020101415A (ja) * 2018-12-20 2020-07-02 パナソニックＩｐマネジメント株式会社 乗員状態検知システム
JP2020142718A (ja) * 2019-03-07 2020-09-10 日本電産モビリティ株式会社 車両制御装置
WO2021122603A1 (en) * 2019-12-20 2021-06-24 Iee International Electronics & Engineering S.A. Method for seat occupancy detection
WO2021240777A1 (ja) * 2020-05-29 2021-12-02 三菱電機株式会社 乗員検出装置および乗員検出方法

Also Published As

Publication number Publication date
JPWO2024121876A1 (https=) 2024-06-13
JP7778250B2 (ja) 2025-12-01

Similar Documents

Publication Publication Date Title
EP3831647B1 (en) Vehicle occupant detection device
US11447085B2 (en) 3D time of flight active reflecting sensing systems and methods
CN106456062B (zh) 用于无人照顾的孩子的检测的方法以及系统
US12145596B2 (en) Cabin-inside detection device and cabin-inside detection method
US10459080B1 (en) Radar-based object detection for vehicles
CN107818277A (zh) 用于监视车辆的车辆内室的方法和设备
US10422874B2 (en) Method and system for spatial modeling of an interior of a vehicle
CN118004093A (zh) 用于监控机动车的方法和机动车
WO2025205258A1 (ja) 検知装置
JP2021122300A (ja) 生体状態検出装置および生体状態検出方法
US12570212B2 (en) Occupant-left-unattended warning device, occupant-left-unattended warning system, and occupant-left-unattended warning method
JP2020056628A (ja) 信号処理装置、センサシステム、警報システム、及び車両
JP7778250B2 (ja) 置き去り判定システム、置き去り判定装置、および、置き去り判定方法
KR102678018B1 (ko) 뒷좌석 승객 기반 차량 제어 장치 및 방법
WO2025099962A1 (ja) 制御装置、制御方法、および、制御システム
JP7851480B2 (ja) 異常検知装置および異常検知方法
CN118196999B (zh) 儿童遗留探测方法、系统、车载设备、儿童座椅及存储介质
WO2025262767A1 (ja) 警報制御装置および警報制御方法
JP2026061441A (ja) 生体検知システムおよび生体検知方法
TW201441076A (zh) 車輛之車門防撞感測器

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22967730

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2024562388

Country of ref document: JP

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22967730

Country of ref document: EP

Kind code of ref document: A1