WO2024134815A1 - 情報処理装置、検知センサ、情報処理システム、および、情報処理方法 - Google Patents

情報処理装置、検知センサ、情報処理システム、および、情報処理方法 Download PDF

Info

Publication number
WO2024134815A1
WO2024134815A1 PCT/JP2022/047262 JP2022047262W WO2024134815A1 WO 2024134815 A1 WO2024134815 A1 WO 2024134815A1 JP 2022047262 W JP2022047262 W JP 2022047262W WO 2024134815 A1 WO2024134815 A1 WO 2024134815A1
Authority
WO
WIPO (PCT)
Prior art keywords
radio wave
error
wave sensor
camera
moving object
Prior art date
Application number
PCT/JP2022/047262
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
浩隆 坂本
匠 武井
靖憲 星原
Original Assignee
三菱電機株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 三菱電機株式会社 filed Critical 三菱電機株式会社
Priority to JP2024565490A priority Critical patent/JPWO2024134815A1/ja
Priority to PCT/JP2022/047262 priority patent/WO2024134815A1/ja
Publication of WO2024134815A1 publication Critical patent/WO2024134815A1/ja

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
    • G01S13/86—Combinations of radar systems with non-radar systems, e.g. sonar, direction finder
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
    • G01S13/88—Radar or analogous systems specially adapted for specific applications
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
    • G01S7/02—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S13/00
    • G01S7/40—Means for monitoring or calibrating
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Definitions

  • This disclosure relates to an information processing device, a detection sensor, an information processing system, and an information processing method.
  • a technique for correcting misalignment of the installation position or orientation of a camera installed in a vehicle cabin in which a camera captures an image of a marker or the like whose direction or distance from the camera is known, and adjusts the installation position or orientation of the camera or corrects the captured image based on the amount of misalignment between the position of the marker or the like in the captured image and the position of the marker or the like that should have been captured in the captured image (for example, Patent Document 1).
  • the sensor when a radio wave sensor is installed in a vehicle cabin, the sensor is fixed in the vehicle cabin via some structure between the sensor and the vehicle frame, such as the ceiling or dashboard. Therefore, there is a possibility that the installation position of the radio wave sensor installed in the vehicle cabin or the direction of the radio waves emitted by the radio wave sensor may be misaligned.
  • the misalignment of the installation position of the radio wave sensor or the direction of the radio waves emitted by the radio wave sensor is referred to as the "installation error of the radio wave sensor.”
  • the installation error of the radio wave sensor As a result, there is a problem that the information output from the radio wave sensor may have been generated when an installation error occurred in the radio wave sensor.
  • Patent Document 1 the technology for adjusting the installation position or orientation of the camera, or correcting the captured image, but does not take into account installation errors of the radio wave sensor, and therefore remains unable to solve the above problem.
  • the present disclosure has been made to solve the problems described above, and aims to provide an information processing device that can provide information (hereinafter referred to as "error-taking information") that can be used to use information generated based on the reflected waves received by a radio wave sensor that is installed in the vehicle cabin and can detect the position and movement of moving objects in the vehicle cabin by emitting radio waves into the vehicle cabin and receiving the reflected waves when the radiated radio waves are reflected by an object, as information obtained without any installation error of the radio wave sensor.
  • error-taking information information generated based on the reflected waves received by a radio wave sensor that is installed in the vehicle cabin and can detect the position and movement of moving objects in the vehicle cabin by emitting radio waves into the vehicle cabin and receiving the reflected waves when the radiated radio waves are reflected by an object, as information obtained without any installation error of the radio wave sensor.
  • the information processing device is provided with an image acquisition unit that acquires an image captured by the camera from a detection sensor having a camera that is provided in the vehicle cabin and captures an image of a target area in the vehicle cabin where at least an occupant may be present, and a radio wave sensor that generates moving object distribution information that represents a three-dimensional distribution of an area in the vehicle cabin where moving objects are present, based on radio waves radiated toward the target area in the vehicle cabin and reflected by objects in the vehicle cabin, a camera error detection unit that detects an installation error of the camera based on the image acquired by the image acquisition unit, a radio wave sensor error calculation unit that calculates an installation error of the radio wave sensor based on the camera installation error detected by the camera error detection unit, and an output unit that outputs error consideration information for converting the moving object distribution information generated by the radio wave sensor into moving object distribution information that takes into account the installation error of the radio wave sensor calculated by the radio wave sensor error calculation unit.
  • a radio wave sensor is installed in the vehicle cabin, emits radio waves into the vehicle cabin, and receives reflected waves from objects to detect the position and movement of moving objects in the vehicle cabin. This information is generated based on the reflected waves received by the radio wave sensor, and can be provided as information obtained without any installation error of the radio wave sensor.
  • FIG. 1 is a diagram illustrating a configuration example of an information processing device according to a first embodiment
  • 1 is a diagram for explaining a configuration example of a detection sensor according to a first embodiment
  • 10A to 10C are diagrams for explaining other configuration examples of the detection sensor according to the first embodiment.
  • 2 is a configuration diagram showing a radio wave sensor mounted on a detection sensor in embodiment 1.
  • FIG. 2 is a block diagram showing the functions of a radio wave sensor mounted on a vehicle in the first embodiment;
  • FIG. 4 is an explanatory diagram showing an example of an FM signal Tx(k) generated by a high-frequency signal generating circuit.
  • FIG. FIG. 2 is an explanatory diagram showing an example of an FM transmission wave and an FM reception wave.
  • FIG. 2 is an explanatory diagram showing the relationship between the sweep time T of an FM transmission wave and the frequency difference fd .
  • FIG. 2 is an explanatory diagram showing an example of a virtual antenna configured in a radio wave sensor.
  • 10A, 10B, and 10C are diagrams showing an example of a moving object map generated by the radio wave sensor in the first embodiment.
  • 4 is a flowchart for explaining an operation of the information processing device according to the first embodiment.
  • Figures 12A and 12B are figures used to explain an example of the effect that would be exerted on a monitoring device that monitors occupants using a moving object map generated by a radio wave sensor if there were a deviation in the position and angle of a moving object detected by the radio wave sensor.
  • FIGS. 13A and 13B are diagrams illustrating an example of a hardware configuration of an information processing device according to the first embodiment.
  • FIG. 11 is a diagram illustrating a configuration example of an information processing device according to a second embodiment.
  • 10 is a flowchart for explaining an operation of an information processing device according to a second embodiment.
  • FIG. 11 is a diagram illustrating a configuration example of an information processing device according to a third embodiment.
  • 13 is a flowchart for explaining an operation of an information processing device according to a third embodiment.
  • Embodiment 1 The information processing device according to the first embodiment provides information (hereinafter referred to as "error-considered information") for a monitoring device that monitors occupants in the vehicle cabin to use the moving object distribution information as information obtained without installation error of the radio wave sensor, based on information (hereinafter referred to as "moving object distribution information") that is generated by a radio wave sensor installed in the vehicle cabin and that represents a three-dimensional distribution of an area in the vehicle cabin where moving objects exist.
  • the radio wave sensor detects moving objects based on reflected waves that are generated by objects in the vehicle cabin after emitting radio waves toward at least an area in the vehicle cabin where occupants may exist (hereinafter referred to as a "target area”), and generates moving object distribution information.
  • the term “radio wave sensor installation error” refers to a deviation in the installation position of the radio wave sensor in the real space inside the vehicle cabin (installation position error), or a deviation in the direction of the radio waves emitted by the radio wave sensor in the real space inside the vehicle cabin (radio wave direction error). More specifically, the term “radio wave sensor installation error” refers to an error in the installation position of the radio wave sensor relative to a predetermined installation position, or an error in the direction of the radio waves emitted by the radio wave sensor relative to a predetermined direction. Note that the "error” referred to here does not include errors that are not problematic in practical use, but refers to an error that has an effect on a monitoring device that performs processing using moving object distribution information generated by the radio wave sensor. In the following description, when the term “direction of the radio wave sensor” is mentioned, this refers to the "direction of the radio waves emitted by the radio wave sensor”.
  • the error-considered information is, in detail, moving object distribution information generated by the radio wave sensor after correcting the moving object distribution information so that the moving object distribution information becomes moving object distribution information obtained without any installation error of the radio wave sensor (hereinafter referred to as "corrected moving object distribution information").
  • the information processing device generates the corrected moving object distribution information and outputs it to the monitoring device.
  • the moving object distribution information is a so-called "moving object map.”
  • the moving object distribution information is also referred to as a moving object map
  • the corrected moving object distribution information is also referred to as a corrected moving object map. Details of the moving object map will be described later.
  • FIG. 1 is a diagram showing an example of a configuration of an information processing device 1 according to the first embodiment.
  • the information processing device 1 is connected to a detection sensor 2 and a monitoring device 3.
  • the information processing device 1 and the detection sensor 2 constitute an information processing system 4.
  • the detection sensor 2 is equipped with a radio wave sensor 21 and a camera 22 .
  • the information processing device 1 corrects the moving object map based on the image captured by the camera 22 mounted on the detection sensor 2 and the moving object map generated by the radio wave sensor 21 mounted on the detection sensor 2, and outputs the corrected moving object map to the monitoring device 3 as error-taking information.
  • the monitoring device 3 includes, for example, an occupant detection device 31 and a physical build determination device 32 , and monitors the occupants in the vehicle cabin based on the corrected moving object map output from the information processing device 1 .
  • the information processing device 1, the detection sensor 2, and the monitoring device 3 are mounted on, for example, a vehicle (not shown).
  • the information processing device 1, the occupant detection device 31, and the physical build determination device 32 will be described in detail later, and first, the detection sensor 2 according to the first embodiment will be described in detail.
  • the detection sensor 2 is provided in the vehicle cabin, for example, in the overhead console. Note that this is merely one example, and the detection sensor 2 may be provided in a location other than the overhead console, such as on the ceiling or dashboard.
  • the detection sensor 2 is provided in a location in the vehicle cabin where the camera 22 can capture an image of the target area and where the radio wave sensor 21 can detect moving objects present in the target area, in other words, where the radio wave sensor 21 and the camera 22 can view the target area in the vehicle cabin. Note that the target area captured by the camera 22 and the target area to which the radio wave sensor 21 emits radio waves do not have to coincide.
  • the detection sensor 2 is installed so that the radio wave sensor 21 emits radio waves in a predetermined direction (hereinafter referred to as the "reference sensor direction") at a predetermined installation position (hereinafter referred to as the "reference sensor position") in the vehicle cabin, and the camera 22 faces a predetermined installation position (hereinafter referred to as the “camera reference position”) and a predetermined direction (hereinafter referred to as the "reference camera direction”).
  • the detection sensor 2 is fixed inside the vehicle cabin via some kind of structure between it and the vehicle frame, such as the dashboard, and therefore there is a possibility that an installation error will occur in the radio wave sensor 21 installed inside the vehicle cabin.
  • the position of the radio wave sensor 21 is represented by the center position of the radio wave sensor 21.
  • the position of the camera 22 is represented by the center position of the camera 22.
  • the positions of the radio wave sensor 21 and the camera 22 are represented in a three-dimensional coordinate system that represents the real space in the vehicle cabin.
  • the x-axis is an axis parallel to the vehicle width direction
  • the y-axis is an axis parallel to the vehicle height direction
  • the z-axis is an axis parallel to the vehicle length direction. Note that in the first embodiment, "parallel” is not limited to being strictly “parallel” and includes approximately parallel.
  • FIGS. 2 and 3 are diagrams for explaining a configuration example of the detection sensor 2 according to the first embodiment. 2 and 3, the upward direction in the drawings corresponds to the upward direction when the detection sensor 2 is installed in the vehicle cabin.
  • FIG. 2 for the sake of convenience, only the radio wave sensor 21, the camera 22, and the fixed part 23 are shown as components of the detection sensor 2.
  • FIG. 3 for the sake of convenience, only the camera 22, the transmitting antenna 11, the receiving antenna 12, and the substrate 24 are shown as components of the detection sensor 2.
  • the detection sensor 2 includes a fixed part 23, a camera 22 provided on the fixed part 23, and a radio wave sensor 21 provided on the fixed part 23.
  • the detection sensor 2 is installed in the vehicle interior, at least two of the three-dimensional coordinate axes of the camera 22 and the three-dimensional coordinate axes of the radio wave sensor 21 are configured to coincide with each other.
  • the three-dimensional coordinate axes of the camera 22 are the x-axis, y-axis, and z-axis specific to the camera 22, and the three-dimensional coordinate axes of the radio wave sensor 21 are the x-axis, y-axis, and z-axis specific to the radio wave sensor 21.
  • the x-axis and y-axis specific to the camera 22 refer to axes on the plane of the image sensor of the camera 22, and the z-axis refers to an axis in the normal direction to the image sensor.
  • the x-axis and y-axis specific to the radio wave sensor 21 refer to axes on the surface of the substrate 24 on which the antennas (the transmitting antenna 11 and the receiving antenna 12) are arranged, and the z-axis refers to an axis in the normal direction to the substrate 24.
  • "match" is not limited to a perfect match, but includes a match within an allowable error range.
  • the detection sensor 2 may, for example, comprise a camera 22, a radio wave sensor 21, and a substrate 24 on which the camera, the transmitting antenna 11 (see Figure 4 described below) of the radio wave sensor 21, and the receiving antenna 12 (see Figure 4 described below) of the radio wave sensor 21 are arranged on a common surface, and the camera 22 may be configured to be arranged between the transmitting antenna 11 and the receiving antenna 12 on the common surface of the substrate 24.
  • the transmitting antenna 11 shown in FIG. 3 is, more specifically, transmitting antenna elements 11-1 and 11-2 (see FIG. 4 described later) that the transmitting antenna 11 has
  • the receiving antenna 12 shown in FIG. 3 is, more specifically, receiving antenna elements 12-1 to 12-4 (see FIG. 4 described later) that the receiving antenna 12 has.
  • FIG. 3 only one transmitting antenna element 11-1 and one receiving antenna element 11-2 are illustrated, but in the radio wave sensor 21, the transmitting antenna 11 may have multiple transmitting antenna elements 11-1 and one receiving antenna element 11-2.
  • the transmitting antenna 11 may have multiple transmitting antenna elements 11-1 and one receiving antenna element 11-2.
  • three receiving antenna elements 12-1 to 12-4 are illustrated, but in the radio wave sensor 21, the receiving antenna 12 has receiving antenna elements 12-1 to 12-4 is not limited to three.
  • the receiving antenna 12 has multiple receiving antenna elements 12-1 to 12-4.
  • the radio wave sensor 21 will be described in detail later.
  • the camera 22 captures an image of a target area within the vehicle cabin.
  • the camera 22 is, for example, a near-infrared camera or a visible light camera.
  • the camera 22 is assumed to be shared with a so-called DMS (Driver Monitoring System) that is installed for the purpose of monitoring the interior of the vehicle.
  • DMS Driver Monitoring System
  • the radio wave sensor 21 generates moving object distribution information, i.e., a moving object map, that represents in three dimensions the distribution of areas where moving objects exist within the vehicle cabin based on the reflected waves generated when radio waves are emitted toward a target area within the vehicle cabin and reflected by objects within the cabin.
  • the radio wave sensor 21 is, for example, a millimeter wave radar.
  • the detection sensor 2 outputs the image captured by the camera 22 and the moving object map generated by the radio wave sensor 21 to the information processing device 1.
  • the radio wave sensor 21 according to the first embodiment will now be described in detail.
  • FIG. 4 is a configuration diagram showing the radio wave sensor 21 mounted on the detection sensor 2 in the first embodiment.
  • FIG. 5 is a block diagram showing the functions of the radio wave sensor 21 mounted on the vehicle in the first embodiment.
  • the x-axis of the three-dimensional coordinate system is parallel to the vehicle width direction
  • the y-axis is parallel to the vehicle height direction
  • the z-axis is parallel to the vehicle length direction.
  • the transmitting antenna 11 is a planar antenna configured on an electronic circuit board.
  • the transmitting antenna 11 has a plurality of transmitting antenna elements 11-1 and 11-2 that radiate radio waves toward a target area.
  • the transmitting antenna element 11-1 and the transmitting antenna element 11-2 are installed at different positions in the vehicle height direction.
  • the installation position of the transmitting antenna element 11-1 in the vehicle height direction is higher than the installation position of the transmitting antenna element 11-2 in the vehicle height direction.
  • the receiving antenna 12 is a planar antenna configured on an electronic circuit board, and is installed on the same plane as the transmitting antenna 11. However, the same plane here does not mean that the plane on which the transmitting antenna 11 is installed and the plane on which the receiving antenna 12 is installed are strictly the same, and also includes different planes as long as there are no practical problems.
  • the receiving antenna 12 has a plurality of receiving antenna elements 12 - 1 to 12 - 4 that receive the reflected waves of the radio waves radiated from the transmitting antenna 11 .
  • receiving antenna element 12-1, receiving antenna element 12-2, receiving antenna element 12-3, and receiving antenna element 12-4 are different from each other in the vehicle width direction.
  • the radio wave sensor circuit section 13 includes a high frequency signal generating circuit 14, a radio wave transmitting section 15, a radio wave receiving section 16, an analog to digital conversion circuit (hereinafter referred to as the “A/D conversion circuit”) 17, a map generating section 18, a communication circuit 19 and a power supply circuit 200.
  • the high frequency signal generating circuit 14 , the radio wave transmitting section 15 and the radio wave receiving section 16 constitute a radio wave transmitting/receiving section 130 that radiates radio waves from the transmitting antenna 11 and obtains a reception signal based on the reflected waves from the receiving antenna 12 .
  • the high frequency signal generating circuit 14 generates an FM (Frequency Modulation) signal, the frequency of which changes over time, as a sensing signal, and outputs the FM signal to both the radio wave transmitting unit 15 and the radio wave receiving unit 16 .
  • the FM-CW (Frequency Modulation-Continuous Wave) method is used as the modulation method, and the high frequency signal generating circuit 14 generates an FM signal.
  • the modulation method is not limited to the FM-CW method, and for example, the FCM (Fast-Chirp Modulation) method may be used.
  • the high frequency signal generating circuit 14 When the FCM method is used as the modulation method, the high frequency signal generating circuit 14 generates an FCM signal and outputs the FCM signal to each of the radio wave transmitting unit 15 and the radio wave receiving unit 16.
  • the radio wave transmitting section 15 has a transmitting circuit 15-1 and a transmitting circuit 15-2.
  • the radio wave transmitting unit 15 causes one of the transmitting antenna elements 11-1 and 11-2 to radiate radio waves toward a target area. That is, the radio wave transmitting unit 15 switches in sequence between the transmitting antenna elements 11-1 and 11-2 to one of the transmitting antenna elements that emits radio waves.
  • the radio wave transmitting unit 15 causes the transmitting antenna element 11-1 to radiate radio waves, it causes the transmitting circuit 15-1 to output an FM signal to the transmitting antenna element 11-1.
  • the radio wave transmitting unit 15 causes the transmitting antenna element 11-2 to radiate radio waves, it causes the transmitting circuit 15-2 to output an FM signal to the transmitting antenna element 11-2.
  • the transmitting circuit 15-1 amplifies the FM signal output from the high frequency signal generating circuit 14 and outputs the amplified FM signal to the transmitting antenna element 11-1, thereby causing the transmitting antenna element 11-1 to radiate an FM transmission wave, which is a radio wave, toward the target area.
  • the transmitting circuit 15-2 amplifies the FM signal output from the high frequency signal generating circuit 14 and outputs the amplified FM signal to the transmitting antenna element 11-2, thereby causing the transmitting antenna element 11-2 to radiate an FM transmission wave toward the target area.
  • the radio wave receiving section 16 has a receiving circuit 16-1, a receiving circuit 16-2, a receiving circuit 16-3 and a receiving circuit 16-4.
  • the receiving circuit 16-m extracts the difference fd between the frequency of the FM signal output from the high frequency signal generating circuit 14 and the frequency of the received signal (hereinafter referred to as the "frequency difference").
  • the receiving circuit 16 - m generates an intermediate frequency signal IF m having a frequency difference f d , and outputs the intermediate frequency signal IF m to the A/D conversion circuit 17 .
  • the A/D conversion circuit 17 outputs the digital signal Dm to the map generation unit 18 .
  • the map generating unit 18 is realized by, for example, a digital signal processing circuit.
  • the map generating unit 18 outputs the generated moving object map to the communication circuit 19 .
  • the digital signal processing circuitry may be implemented, for example, by a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a combination thereof.
  • the digital signal processing circuit is not limited to being realized by dedicated hardware, but may be realized by software, firmware, or a combination of software and firmware.
  • the software or firmware is stored as a program in the memory of a computer.
  • the computer means hardware that executes the program, and includes, for example, a CPU (Central Processing Unit), a central processing unit, a processing unit, an arithmetic unit, a microprocessor, a microcomputer, a processor, or a DSP (Digital Signal Processor).
  • FIG. 6 is an explanatory diagram showing an example of the FM signal Tx(k) generated by the high frequency signal generating circuit 14. As shown in FIG. In the FM signal Tx(k) shown in FIG.
  • the initial state of the frequency is the lower limit frequency, and the frequency increases with time until it reaches the upper limit frequency.
  • the sweep time T is the time it takes for the frequency of the FM signal Tx(k) to reach the upper limit frequency from the lower limit frequency
  • the frequency bandwidth BW is the difference frequency between the upper limit frequency and the lower limit frequency.
  • an FM signal Tx(k) is generated K times at a constant period Tc .
  • the high frequency signal generating circuit 14 outputs an FM signal Tx(k) to transmitting circuits 15-1 and 15-2 of the radio wave transmitting unit 15, and outputs an FM signal Tx(k) to receiving circuits 16-1 to 16-4 of the radio wave receiving unit 16.
  • FIG. 6 also shows the amplitude waveform of the FM signal Tx(k).
  • the transmission circuits 15-1 and 15-2 alternately perform the operation of outputting the FM signal Tx(k).
  • the transmission circuit 15-1 receives the FM signal Tx(k) from the high frequency signal generating circuit 14 at the timing for outputting the FM signal Tx(k)
  • the transmission circuit 15-1 amplifies the FM signal Tx(k) and outputs the amplified FM signal Tx(k) to the transmitting antenna element 11-1.
  • the transmission circuit 15-1 outputs the amplified FM signal Tx(k) to the transmission antenna element 11-1, and an FM transmission wave is radiated from the transmission antenna element 11-1 towards the target area.
  • the transmission circuit 15-2 stops outputting the FM signal Tx(k).
  • the transmission circuit 15-2 When the transmission circuit 15-2 receives the FM signal Tx(k) from the high frequency signal generating circuit 14 at the timing for outputting the FM signal Tx(k), the transmission circuit 15-2 amplifies the FM signal Tx(k) and outputs the amplified FM signal Tx(k) to the transmitting antenna element 11-2. The amplified FM signal Tx(k) is output from the transmission circuit 15-2 to the transmission antenna element 11-2, and an FM transmission wave is radiated from the transmission antenna element 11-2 towards the target area. When the transmission circuit 15-2 is outputting the FM signal Tx(k), the transmission circuit 15-1 stops outputting the FM signal Tx(k).
  • the transmitting antenna element 11-1 When the transmitting antenna element 11-1 receives the FM signal Tx(k) from the transmitting circuit 15-1, it radiates an FM transmission wave whose frequency transitions over time, as shown in FIG. 7, toward the target area.
  • the transmitting antenna element 11-2 receives the FM signal Tx(k) from the transmitting circuit 15-2, it radiates an FM transmission wave as shown in FIG. 7 toward the target area.
  • FIG. 7 is an explanatory diagram showing an example of an FM transmission wave and an FM reception wave. The FM transmission waves radiated from the transmitting antenna elements 11-1 and 11-2 are reflected by passengers and the like present in the target area.
  • the FM received waves which are reflected waves of the FM transmitted waves reflected by passengers or the like, are received by the receiving antenna elements 12-1 to 12-4 after a time td has elapsed since the FM transmitted waves were emitted from the transmitting antenna element 11-1 or the transmitting antenna element 11-2.
  • a time td has elapsed since the FM transmitted waves were emitted from the transmitting antenna element 11-1 or the transmitting antenna element 11-2.
  • the frequency difference fd between the FM transmission wave which is a chirp signal
  • the FM reception wave increases in proportion to the distance between the radio wave sensor 21 and a reflecting object such as a passenger.
  • FIG. 8 is an explanatory diagram showing the relationship between the sweep time T of the FM transmission wave and the frequency difference fd .
  • the receiving circuit 16-m extracts a frequency difference f d (k), which is the difference between the frequency of the FM signal Tx (k) output from the high frequency signal generating circuit 14 and the frequency of the receiving signal Rx(k) output from the receiving antenna element 12-m.
  • the receiving circuit 16 - m generates an intermediate frequency signal IF m (k) having a frequency difference f d (k), and outputs the intermediate frequency signal IF m (k) to the A/D conversion circuit 17 .
  • the signal strength of the beat frequency Fs 1,n becomes higher than the signal strength of frequencies other than the beat frequency Fs 1,n , and becomes the peak value of the first frequency spectrum Sp 1 .
  • Rn is the distance between the radio wave sensor 21 and the passenger or the like
  • c is the propagation speed of the radio wave.
  • the map generator 18 searches the first frequency spectrum Sp1 for a beat frequency Fs1 ,n at which the signal strength reaches a peak value.
  • the map generating unit 18 calculates the distance R n in the kth transmission and reception by substituting the found beat frequency Fs 1,n into equation (1).
  • f 0 is the center frequency of the FM signal Tx(k).
  • the map generator 18 searches the second frequency spectrum Sp2 for a beat frequency Fs2 ,n at which the signal strength reaches a peak value.
  • the map generator 18 calculates the relative velocity v n in the kth transmission and reception by substituting the found beat frequency Fs 2,n into equation (2).
  • dx is the distance between the receiving antenna elements 12-1 to 12-4 in the direction parallel to the vehicle width direction
  • ⁇ x ,n is the angle of incidence of the FM received wave on the xz plane with respect to the receiving antenna element 12-m
  • ⁇ is the wavelength of the FM transmitted wave.
  • the map generator 18 calculates a third frequency spectrum Sp 3,m by combining the results of the Fourier transform of the four digital signals D 1 (k) to D 4 (k).
  • Each time the map generating unit 18 calculates the third frequency spectrum Sp3 ,m it calculates a fourth frequency spectrum Sp4,m by performing a Fourier transform on the third frequency spectrum Sp3, m in the time direction of the FM transmission wave that is periodically radiated.
  • the map generating unit 18 calculates the fourth frequency spectrum Sp4 ,m , it calculates a fifth frequency spectrum Sp5 by performing a Fourier transform on the fourth frequency spectrum Sp4 ,m .
  • the signal strength of the frequency component Fs5 ,n becomes higher than the signal strength of frequency components other than the frequency component Fs5 ,n , and becomes the peak value of the fifth frequency spectrum Sp5 .
  • the map generator 18 searches the fifth frequency spectrum Sp5 for a frequency component Fs5,n at which the signal strength reaches a peak value.
  • the map generating unit 18 calculates the incident angle ⁇ x,n in the kth transmission and reception on the xz plane by substituting the searched frequency component Fs 5,n into the following equation (4).
  • the radio wave sensor 21 shown in Fig. 4 has four receiving antenna elements 12-1 to 12-4.
  • the radio wave sensor 21 also has two transmitting antenna elements 11-1 and 11-2, and the transmitting antenna elements 11-1 and 11-2 alternately radiate FM transmission waves. Therefore, the radio wave sensor 21 configures virtual antennas 12-1' to 12-8' that are equivalent to the four receiving antenna elements 12-1 to 12-4 arranged in two rows in the vehicle height direction, as shown in Fig. 9.
  • virtual antennas 12-1' to 12-4' and virtual antennas 12-5' to 12-8' are aligned in the vehicle height direction.
  • the distance between the virtual antennas 12-1' to 12-4' and the virtual antennas 12-5' to 12-8' in the vehicle height direction is dy .
  • FIG. 9 is an explanatory diagram showing an example of the virtual antennas 12-1' to 12-8' configured in the radio wave sensor 21. As shown in FIG.
  • the map generation unit 18 calculates an incident angle ⁇ y,n in the k-th transmission and reception on the y-z plane by substituting a frequency component Fs 5 , n obtained by performing a Fourier transform on the fourth frequency spectrum Sp 4, m across the vehicle height direction (direction parallel to the y-axis), which is the direction in which the virtual antennas 12-1′, etc. and the virtual antennas 12-5 ′, etc. are lined up, into the following equation (5).
  • the map generating unit 18 generates a three-dimensional spatial distribution having dimensions of the distance R n , the angle of incidence ⁇ x,n on the x-z plane, and the angle of incidence ⁇ y,n on the y-z plane.
  • the process of generating the three-dimensional spatial distribution is a known technique, and therefore a detailed description thereof will be omitted.
  • the three-dimensional spatial distribution generated by the map generating unit 18 is a "moving object map.”
  • the moving object map represents the distribution of the area in the vehicle cabin where the moving objects exist in three dimensions.
  • the moving object map represents the minute movements of the moving objects, in other words, the objects that have reflected the radio waves irradiated by the radio wave sensor 21, in the vehicle cabin, using a plurality of grids that correspond to the reflection points of the radio waves in the three-dimensional space.
  • FIGS. 10A, 10B, and 10C are diagrams showing an example of a moving object map generated by the radio wave sensor 21 in the first embodiment.
  • Fig. 10A shows an example of a top view of the moving object map
  • Fig. 10B shows an example of a side view of the moving object map
  • Fig. 10C shows an example of a front view of the moving object map.
  • Fig. 10A, Fig. 10B, and Fig. 10C show the moving object map in three views.
  • the moving object maps shown in Fig. 10A, Fig. 10B, and Fig. 10C are moving object maps when an occupant is present in the rear right seat in the vehicle cabin.
  • a numerical value according to the speed of an object existing at the position of the grid is assigned to each grid.
  • the faster the speed of an object existing at the position of a certain grid the larger the numerical value assigned to that grid.
  • a grid included in a range where a moving object with slight movement, in other words, an occupant is present is assigned a larger numerical value than a grid where no occupant is present.
  • the grids to which larger numerical values are assigned are shown darker.
  • the grids that are included in the range where an occupant is present are shown darker.
  • stationary objects in other words, reflected wave components with a velocity of zero, do not appear in the moving object map.
  • each grid in the moving object map is associated with coordinates in the real space inside the vehicle cabin, which is expressed in a three-dimensional coordinate system representing the real space inside the vehicle cabin.
  • the map generating unit 18 assumes that the radio wave sensor 21 is installed at a reference sensor position, and associates each grid with coordinates in the real space inside the vehicle cabin based on the reference position.
  • the communication circuit 19 transfers the moving object map output from the map generating unit 18 to the information processing device 1 via the interface unit 210 described later.
  • the power supply circuit 200 receives power from a control unit (not shown) or the like via an interface unit 210 .
  • the power supply circuit 200 distributes the received power as driving power to the high frequency signal generating circuit 14, the radio wave transmitting section 15, the radio wave receiving section 16, the A/D conversion circuit 17, the map generating section 18 and the communication circuit 19.
  • the interface unit 210 is an interface for connecting the radio wave sensor circuit unit 13 to the information processing device 1 or a control unit (not shown) or the like.
  • the radio wave sensor 21 in embodiment 1 is a radio wave sensor 21 capable of detecting the distance and angle to an object present in the vehicle cabin, and whether the object is a moving object, and is assumed to be a radio wave sensor capable of generating a moving object map based on the distance and angle to the detected object (more specifically, the moving object).
  • the receiving antenna 12 in the radio wave sensor 21 has four receiving antenna elements 12-1 to 12-4. However, this is merely an example.
  • the receiving antenna 12 may have a plurality of receiving antenna elements, and the receiving antenna 12 may have two receiving antenna elements, three receiving antenna elements, or five or more receiving antenna elements.
  • the transmitting antenna 11 in the radio wave sensor 21 has two transmitting antenna elements 11-1 and 11-2. However, this is merely an example. In the radio wave sensor 21, the transmitting antenna 11 may have only one transmitting antenna element, or may have three or more transmitting antenna elements.
  • the information processing device 1 includes a moving object distribution information acquisition unit 101, an image acquisition unit 102, a camera error detection unit 103, a radio wave sensor error calculation unit 104, a correction unit 105, and an output unit 106.
  • the moving object distribution information acquisition unit 101 acquires moving object distribution information generated by the radio wave sensor 21, that is, a moving object map, from the detection sensor 2.
  • the moving object distribution information acquisition unit 101 outputs the acquired moving object map to the correction unit 105 .
  • the image acquisition unit 102 acquires the captured image captured by the camera 22 from the detection sensor 2 .
  • the image acquisition unit 102 outputs the acquired captured image to the camera error detection unit 103 .
  • the camera error detection unit 103 detects an installation error of the camera 22 based on the captured image acquired by the image acquisition unit 102 .
  • the term "installation error of camera 22" refers to a deviation in the installation position of camera 22 in the real space inside the vehicle cabin (installation position error) or a deviation in the orientation of camera 22 in the real space inside the vehicle cabin (orientation error of camera 22). More specifically, the term “installation error of camera 22” refers to an error in the installation position of camera 22 with respect to a predetermined installation position (hereinafter referred to as a "reference camera position”), or an error in the orientation of camera 22 with respect to a predetermined orientation (hereinafter referred to as a "reference camera orientation"). More specifically, the error in the installation position of the camera 22 refers to an error in the vehicle width direction, the vehicle height direction, and the vehicle length direction of the installation position of the camera 22 within the vehicle cabin.
  • the camera error detection unit 103 uses a known image recognition technique to compare the position and size of a predetermined target object on the captured image with the position (hereinafter referred to as the "target position") and size (hereinafter referred to as the "target size") of the target object on the captured image where the target object should be captured, thereby detecting an error in the installation position of the camera 22 among the installation errors of the camera 22.
  • the target object may be, for example, a structure in the vehicle cabin, such as a window frame, a vehicle body, or a seat frame in the vehicle cabin, or may be a marker that is attached in advance to the vehicle cabin by printing or the like.
  • the target position and target size of the target on the captured image are, in detail, the position and size of the target on the captured image captured by camera 22 when camera 22 is installed at the reference camera position, in the reference camera orientation, and without any misalignment.
  • an administrator or the like may obtain information on the target position and target size of the target object by conducting a test in advance, and store the obtained information on the target position and target size in the camera error detection unit 103 in association with information indicating the target object.
  • the target position and target size are represented by coordinates on the captured image.
  • the target position is represented, for example, by the coordinates of the center of the target object in the captured image.
  • the target size is represented, for example, by the coordinates of the four corners of the smallest rectangle that surrounds the target object in the captured image.
  • the camera error detection unit 103 can detect an error in the vehicle width direction in the installation position of the camera 22 in the real space based on the difference in the vehicle width direction between the position of the target captured in the captured image acquired by the image acquisition unit 102 and the target position. Note that, for example, a calculation formula for determining the difference in the vehicle width direction between the position of the target and the target position in the real space when the difference in the vehicle width direction between the position of the target and the target position in the captured image is large is defined in advance and stored in the camera error detection unit 103.
  • the camera error detection unit 103 can detect an error in the vehicle height direction in the installation position of the camera 22 in real space based on the difference in the vehicle height direction between the position of the target object captured in the captured image acquired by the image acquisition unit 102 and the target position. Note that, for example, a calculation formula for determining the difference in the vehicle height direction between the position of the target object and the target position in real space when the difference in the vehicle height direction between the position of the target object and the target position in the captured image is large is defined in advance and stored in the camera error detection unit 103.
  • the camera error detection unit 103 can detect an error in the vehicle length direction in real space in the installation position of the camera 22 based on the difference between the size of the target captured in the captured image acquired by the image acquisition unit 102 and the target size. Note that a calculation formula for determining the difference in the vehicle length direction between the target position and the target position in real space, for example, when the difference in the vehicle length direction between the target position and the target position in the captured image is about is predefined and stored in the camera error detection unit 103.
  • the camera error detection unit 103 can detect an error in the orientation of the camera 22, which is one of the installation errors of the camera 22, based on the difference between the positions of multiple targets captured in the captured image acquired by the image acquisition unit 102 and the target positions of the multiple targets.
  • the camera error detection unit 103 may detect an error in the installation position of the camera 22 and an error in the orientation of the camera 22, i.e., an installation error of the camera 22, based on the captured image acquired by the image acquisition unit 102, information indicating the target object, the target position, the target size, and a trained model (hereinafter referred to as the "machine learning model").
  • the machine learning model is a model that receives the captured image, information indicating the target object, the target position, and the target size as input, and outputs an error in the installation position of the camera 22 and an error in the orientation of the camera 22.
  • the camera error detection unit 103 outputs information regarding the detected installation error of the camera 22, i.e., information regarding the error in the installation position of the camera 22 and the error in the orientation of the camera 22 (hereinafter referred to as ⁇ camera installation error information''), to the radio sensor error calculation unit 104.
  • the installation position error of the camera 22 is represented, for example, by distances in the real space inside the vehicle cabin (distance in the vehicle width direction, distance in the vehicle height direction, and distance in the vehicle length direction).
  • the orientation error of the camera 22 is represented, for example, by an angle indicating the difference in the orientation of the camera 22 from the reference camera orientation.
  • the radio wave sensor error calculation unit 104 calculates the installation error of the radio wave sensor 21 based on the installation error of the camera 22 detected by the camera error detection unit 103 .
  • the "installation error of radio wave sensor 21” refers to an error in the installation position of radio wave sensor 21 relative to a predetermined specified installation position (hereinafter referred to as the "reference sensor position") in the real space within the vehicle cabin, or an error in the orientation of radio wave sensor 21 relative to a predetermined specified orientation (hereinafter referred to as the "reference sensor orientation"). More specifically, the error in the installation position of the radio wave sensor 21 refers to an error in the vehicle width direction, vehicle height direction, and vehicle length direction of the installation position of the radio wave sensor 21 within the vehicle cabin.
  • the radio wave sensor error calculation unit 104 can calculate the installation error of the radio wave sensor 21 based on the installation error of the camera 22.
  • the radio wave sensor error calculation unit 104 may calculate the installation error of the radio wave sensor 21, for example, using a table in which the installation error of the camera 22 and the installation error of the radio wave sensor 21 are associated (hereinafter referred to as the “radio wave sensor error calculation table”).
  • the radio wave sensor error calculation table is generated in advance by an administrator or the like and stored in the radio wave sensor error calculation unit 104 .
  • the radio wave sensor error calculation unit 104 outputs the calculated information relating to the installation error of the radio wave sensor 21 (hereinafter referred to as “sensor installation error information”) to the correction unit 105 .
  • the installation position error of the radio wave sensor 21 is represented, for example, by distances in the real space inside the vehicle cabin (distance in the vehicle width direction, distance in the vehicle height direction, and distance in the vehicle length direction).
  • the orientation error of the radio wave sensor 21 is represented, for example, by the difference in the orientation of the radio wave sensor 21 with respect to the orientation of a reference sensor.
  • the correction unit 105 corrects the moving object map acquired by the moving object distribution information acquisition unit 101 based on the installation error of the radio wave sensor 21 calculated by the radio wave sensor error calculation unit 104 .
  • the correction unit 105 first calculates parameters for correcting the moving object map (hereinafter, referred to as “map correction parameters”) based on the installation error of the radio wave sensor 21 .
  • the map correction parameters are parameters that define the amount of rotation or horizontal movement of the moving body map depending on, for example, the error in the installation position of the radio wave sensor 21 (i.e., the error in the installation position of the radio wave sensor 21 in the vehicle width direction, the error in the vehicle height direction, and the error in the vehicle length direction) and the error in the orientation of the radio wave sensor 21 (the difference in orientation from the reference sensor orientation).
  • the map correction parameters may be, for example, a table for converting coordinates associated with each grid in the moving object map (hereinafter, referred to as a "map correction table")
  • the coordinates associated with each grid are, for example, coordinates in the real space inside the vehicle cabin.
  • the map correction table is information that corresponds to the error in the installation position of the radio wave sensor 21 (i.e., the error in the installation position of the radio wave sensor 21 in the vehicle width direction, the error in the vehicle height direction, and the error in the vehicle length direction), the error in the orientation of the radio wave sensor 21 (the difference in orientation from the reference sensor orientation), the amount of movement of which grid of the moving object map should be moved in the vehicle width direction, vehicle height direction, and vehicle length direction (e.g., how many pixels or how many millimeters), and the angle by which the moving object map should be rotated after the movement.
  • the error in the installation position of the radio wave sensor 21 i.e., the error in the installation position of the radio wave sensor 21 in the vehicle width direction, the error in the vehicle height direction, and the error in the vehicle length direction
  • the error in the orientation of the radio wave sensor 21 the difference in orientation from the reference sensor orientation
  • Multiple map correction tables may be prepared, for example, depending on the type of installation error of the radio wave sensor 21.
  • multiple map correction tables may be prepared, such as a map correction table for when there is an error in the installation position of the radio wave sensor 21 only in the vehicle width direction, a map correction table for when there is an error in the vehicle height direction only, and a map correction table for when there is an error in the vehicle length direction only.
  • the correction unit 105 determines which map correction table to use to correct the moving body map, depending on the type of installation error of the radio wave sensor 21.
  • the correction unit 105 outputs the corrected moving object map to the output unit 106 .
  • the correction unit 105 outputs the moving object map acquired by the moving object distribution information acquisition unit 101 as a corrected moving object map to the output unit 106 without any change.
  • the output unit 106 outputs the corrected moving object map output from the correction unit 105, i.e., the moving object map corrected by the correction unit 105, to the monitoring device 3 as error consideration information.
  • the monitoring device 3 monitors the occupants in the vehicle cabin based on the moving object map output from the information processing device 1, specifically, based on the corrected moving object map.
  • the monitoring device 3 includes an occupant detection device 31 and a physical size determination device 32 .
  • the occupant detection device 31 detects whether or not a child has been left behind inside the vehicle cabin based on the corrected moving object map.
  • a child is assumed to be an occupant whose physique would make it difficult for them to get out of the vehicle on their own if left behind.
  • children include infants.
  • the occupant detection device 31 first detects an occupant present in the vehicle compartment and determines whether the occupant is an adult or a child. Specifically, the occupant detection device 31 determines whether or not the spatial distribution included in the corrected moving object map (for example, the spatial distribution shown by the colored grid in Figure 10) is a spatial distribution corresponding to an occupant (hereinafter referred to as an ⁇ occupant spatial distribution'') based on the shape of the spatial distribution. Any method can be used for the determination based on the shape of the spatial distribution, but one possible method is to use a model that has learned the shape of the occupant spatial distribution to determine whether the spatial distribution included in the moving object map is an occupant spatial distribution.
  • the occupant detection device 31 may determine that the spatial distribution is an occupant spatial distribution.
  • a preset threshold hereinafter referred to as the "speed determination threshold”
  • the speed determination threshold is stored in a location that can be referenced by the occupant detection. Even if the occupant is not moving their arms or legs, they are still breathing and therefore have some movement. If the occupant detection device 31 determines that the spatial distribution included in the corrected moving object map is an occupant spatial distribution, it detects that an occupant is present in the vehicle cabin.
  • the occupant detection device 31 determines that the spatial distribution included in the corrected moving object map is an occupant spatial distribution, it identifies the position of the occupant present in the occupant spatial distribution from the distance R n , incidence angle ⁇ x,n and incidence angle ⁇ y, n related to the occupant spatial distribution.
  • the occupant detection device 31 can identify the position of the occupant if the distance R n from the radio wave sensor 21 and the incidence angle ⁇ x,n and incidence angle ⁇ y,n with respect to the radio wave sensor 21 are known.
  • each grid of the corrected moving object map is associated with the distance R n from the radio wave sensor 21 and the incidence angle ⁇ x,n and incidence angle ⁇ y,n with respect to the radio wave sensor 21.
  • the seating position of the occupant is indicated, for example, by the seat (e.g., rear right seat, rear left seat, or rear center seat).
  • the occupant detection device 31 determines whether the detected occupant is an adult or a child for each seat based on the size of the occupant spatial distribution included in the corrected moving object map. Any method may be used for the determination based on the size of the occupant space distribution. For example, a virtual area for determining physique (hereinafter referred to as "physique determination area") in a pre-set space in the vehicle cabin may be used to determine whether the occupant is an adult or a child.
  • the physique determination area is set in advance by an administrator or the like and stored in a location that can be referenced by the monitoring device 3.
  • the physique determination area is, for example, a rectangular area on the seat surface of each seat, with a rectangle including the seat surface as its bottom surface and a height from the seat surface to the top end of the headrest.
  • the occupant detection device 31 determines whether the occupant is an adult or a child, for example, based on how high the occupant space distribution is in the physique determination area.
  • Another possible method of determination based on the size of the occupant space distribution is to determine whether the occupant is an adult or a child using a model that has learned the size of the occupant space distribution for adults and children.
  • the occupant detection device 31 detects whether or not a child has been left behind in the vehicle compartment based on the result of determining whether the detected occupant is an adult or a child. For example, when the occupant detection device 31 determines that there is no adult present in the vehicle cabin and that there is a child present, i.e., when it determines that only a child is present in the vehicle cabin, it detects that a child has been left behind in the vehicle cabin. For example, when an adult is present in the vehicle compartment, or when neither an adult nor a child is present, the occupant detection device 31 detects that no child has been left behind in the vehicle compartment.
  • the output device is, for example, a mobile terminal carried by the vehicle owner, or a horn or light provided in the vehicle.
  • the occupant detection device 31 when the occupant detection device 31 detects an occupant present in the vehicle cabin, it may cause the output device to output a sound or the like encouraging the occupant to fasten their seat belt.
  • the physique determining device 32 determines the physique of an occupant present in the vehicle cabin based on the corrected moving object map. For example, the physique of the occupant determined by the physique determining device 32 is either "adult” or "child.” In this case, the physical size determination device 32 determines the physical size of the occupant in a manner similar to the manner in which the above-mentioned occupant detection device 31 detects an occupant present in the vehicle cabin and determines whether the occupant is an adult or a child.
  • the physique determination device 32 may further classify the occupant's physique into categories such as "large adult,””smalladult,” or "child.”
  • the physique determination region is divided into three regions in the height direction. If the three regions are, in order from the highest region, a first physique determination region, a second physique determination region, and a third physique determination region, the physique determination device 32 determines whether the occupant spatial distribution included in the corrected dynamic body map is included in the first physique determination region, the second physique determination region, or the third physique determination region. For example, when the occupant spatial distribution is included in all of the first physique determination region, the second physique determination region, and the third physique determination region, the physique determination device 32 determines that the occupant is a "large adult". For example, when the occupant spatial distribution is included only in the third physique determination region, the physique determination device 32 determines that the occupant is a "child".
  • the physique determination device 32 outputs information relating to the determined physique of the occupant (hereinafter referred to as the "physique determination result") to, for example, a seat belt control device (not shown) and a seat control device (not shown).
  • the seat belt control device controls, for example, the amount of withdrawal of the seat belt based on the physical build determination result output from the physical build determination device 32.
  • the seat control device adjusts, for example, the seat position or the backrest based on the physical build determination result output from the physical build determination device 32.
  • FIG. 11 is a flowchart for explaining the operation of the information processing device 1 according to the first embodiment.
  • the information processing device 1 receives an operation instruction
  • the information processing device 1 performs an operation as shown in the flowchart of Fig. 11.
  • a control unit (not shown) of the information processing device 1 receives the operation instruction, and when the control unit receives the operation instruction, the control unit operates each component included in the information processing device 1.
  • the operation instruction is input, for example, by an administrator etc.
  • the administrator etc. inputs the operation instruction, for example, when installing the detection sensor 2 in the vehicle.
  • the moving object distribution information acquisition unit 101 acquires moving object distribution information generated by the radio wave sensor 21, that is, a moving object map, from the detection sensor 2 (step ST1).
  • the moving object distribution information acquisition unit 101 outputs the acquired moving object map to the correction unit 105 .
  • the image acquisition unit 102 acquires the captured image captured by the camera 22 from the detection sensor 2 (step ST2).
  • the image acquisition unit 102 outputs the acquired captured image to the camera error detection unit 103 .
  • the camera error detection unit 103 detects an installation error of the camera 22 based on the captured image acquired by the image acquisition unit 102 in step ST2 (step ST3).
  • the camera error detection unit 103 outputs the camera installation error information to the radio wave sensor error calculation unit 104 .
  • the radio wave sensor error calculation unit 104 calculates the installation error of the radio wave sensor 21 based on the installation error of the camera 22 detected by the camera error detection unit 103 in step ST3 (step ST4).
  • the radio wave sensor error calculation unit 104 outputs the sensor installation error information to the correction unit 105 .
  • the correction unit 105 corrects the moving object map acquired by the moving object distribution information acquisition unit 101 based on the installation error of the radio wave sensor 21 calculated by the radio wave sensor error calculation unit 104 in step ST4 (step ST5).
  • the correction unit 105 outputs the corrected moving object map to the output unit 106 .
  • the correction unit 105 outputs the moving object map acquired by the moving object distribution information acquisition unit 101 as a corrected moving object map to the output unit 106 without any change.
  • the output unit 106 outputs the corrected moving object map output from the correction unit 105 in step ST5, i.e., the moving object map corrected by the correction unit 105, as error consideration information to the monitoring device 3 (step ST6).
  • step ST1 is performed before the processing of step ST2, but this is merely an example.
  • the processing of step ST1 and the processing of step ST2 may be performed in parallel. It is sufficient that the processing of step ST1 is performed before the processing of step ST5 is performed.
  • the information processing device 1 acquires an image captured by the camera 22 from the detection sensor 2, and detects an installation error of the camera 22 based on the captured image.
  • the information processing device 1 detects an installation error of the camera 22, it calculates an installation error of the radio wave sensor 21 based on the installation error of the camera 22.
  • the information processing device 1 corrects the moving object distribution information generated by the radio wave sensor 21 acquired from the detection sensor 2, i.e., the moving object map, based on the calculated installation error of the radio wave sensor 21.
  • the information processing device 1 outputs the corrected moving object map to the monitoring device 3 as error consideration information.
  • the monitoring device 3 monitors the interior of the vehicle cabin based on the corrected moving object map output from the information processing device 1.
  • the radio wave sensor 21 when the radio wave sensor 21 is installed in the vehicle cabin, it is generally fixed in the vehicle cabin via some structure between it and the vehicle frame, such as the ceiling or dashboard. For this reason, there is a possibility that an installation error occurs in the radio wave sensor 21 installed in the vehicle cabin. If an installation error occurs in the radio wave sensor 21, a deviation will also occur in the position and angle of the moving object detected by the radio wave sensor 21. In particular, the position of the body or head of an occupant in a seat located far from the radio wave sensor 21 will be significantly shifted relative to the radio wave sensor 21, and the deviation in the position and angle of the occupant detected by the radio wave sensor 21 will be large. The deviation in the position and angle of the moving object detected by the radio wave sensor 21 has a significant impact on the monitoring results of the monitoring device 3, which monitors the occupants using the moving object map generated by the radio wave sensor 21.
  • Figures 12A and 12B are figures for explaining an example of the effect on the monitoring device 3, which monitors occupants using a moving object map generated by the radio wave sensor 21, if there is a deviation in the position and angle of the moving object detected by the radio wave sensor 21.
  • Figures 12A and 12B show, as an example, a case in which a shift in the orientation of the radio wave sensor 21 causes a shift in the position and angle of the moving object detected by the radio wave sensor 21, thereby affecting the monitoring device 3.
  • the monitoring device 3 is a physical build determination device 32, which uses the physical build determination area to determine whether the occupant's physical build is a "large adult,” a "small adult,” or a “child.”
  • 12A and 12B the physique determination region is indicated by “S.”
  • the vehicle is indicated by "C”
  • the radiation range of the radio wave sensor 21 is indicated by “R.”
  • 12A and 12B are side views of the vehicle.
  • FIG. 12A shows an example of the state inside the vehicle cabin when the radio wave sensor 21 is installed at the same position as the reference sensor position so as to radiate radio waves in the direction of the reference sensor.
  • FIG. 12B shows an example of the state inside the vehicle cabin when the position of the radio wave sensor 21 is correct as the reference sensor position, but the orientation of the radio wave sensor 21 is shifted upward from the reference sensor orientation.
  • the physique determination device 32 would erroneously determine the physique of the occupant. This is because the coordinates in the real space in the vehicle cabin that correspond to each grid on the dynamic body map are shifted by the amount of the deviation.
  • the physique determination area is set according to the coordinates in the real space in the vehicle cabin that correspond to each grid on the dynamic body map, the physique determination area would have to be set in a range such as that shown in FIG. 12B. However, the physique determination area is actually set based on coordinates in the real space of the vehicle interior without any deviation (for example, FIG. 12A ). As a result, the physique determination device 32 may erroneously determine the physique of the occupant, for example, determining the physique of the occupant to be a "small adult" when the physique should be determined to be a "child.”
  • the information processing device 1 acquires an image captured by the camera 22 from the detection sensor 2, and detects an installation error of the camera 22 based on the captured image.
  • the information processing device 1 detects an installation error of the camera 22, it calculates an installation error of the radio sensor 21 based on the installation error of the camera 22.
  • the information processing device 1 corrects the moving object map generated by the radio sensor 21 acquired from the detection sensor 2, based on the calculated installation error of the radio sensor 21.
  • the information processing device 1 outputs the corrected moving object map to the monitoring device 3 as error consideration information.
  • the information processing device 1 can provide error-taking information, specifically, a corrected moving object map, to use the moving object map information generated by the radio wave sensor 21 as information obtained without any installation error of the radio wave sensor 21.
  • the monitoring device 3 can monitor the occupants using the corrected moving object map provided by the information processing device 1, and as a result, the monitoring device 3 can prevent erroneous monitoring of occupants (for example, erroneous detection of a child being left behind, or erroneous judgment of the occupant's physique).
  • the radio wave sensor 21 is mounted on the detection sensor 2, and the detection sensor 2 includes a fixed part 23, a camera 22 provided on the fixed part 23, and the radio wave sensor 21 provided on the fixed part 23.
  • the detection sensor 2 has at least two of the x-axis, y-axis, and z-axis specific to the camera 22 coincide with the x-axis, y-axis, and z-axis specific to the radio wave sensor 21, or the detection sensor 2 includes the camera 22, the radio wave sensor 21, and a substrate 24 on which the transmitting antenna 11 and receiving antenna 12 of the camera 22 and the radio wave sensor 21 are provided on a common surface.
  • the information processing device 1 acquires an image captured by the camera 22 from such a detection sensor 2, and detects an installation error of the camera 22 based on the captured image.
  • the information processing device 1 detects an installation error of the camera 22, it calculates an installation error of the radio wave sensor 21 based on the installation error of the camera 22.
  • the information processing device 1 corrects the moving object map acquired from the detection sensor 2 and generated by the radio wave sensor 21 based on the calculated installation error of the radio wave sensor 21. Then, the information processing device 1 outputs the corrected moving object map to the monitoring device 3 as error consideration information.
  • the information processing device 1 can provide error-taking information, specifically, a corrected moving object map, to use the moving object map information generated by the radio wave sensor 21 as information obtained without any installation error of the radio wave sensor 21.
  • the monitoring device 3 can monitor the occupants using the corrected moving object map provided by the information processing device 1, and as a result, the monitoring device 3 can prevent erroneous monitoring of occupants (for example, erroneous detection of a child being left behind, or erroneous judgment of the occupant's physique).
  • a technology is generally known in which a marker whose exact direction or distance from the camera 22 is known is captured by the camera 22, and a deviation from the position and capturing direction that should have been captured on the captured image is detected, and the installation position and orientation of the camera 22 are adjusted or the captured image is corrected.
  • the radio wave sensor 21 it is not impossible to arrange an object serving as a marker whose exact direction or distance from the radio wave sensor 21 is known, detect the object, detect the position of the object that should be detected and the deviation from the direction in which the object is located, and adjust the installation position of the radio wave sensor 21 and the direction of radio wave emission.
  • the radio wave sensor 21 is difficult for the radio wave sensor 21 to distinguish between the marker and structures and the like in the vehicle interior other than the marker.
  • a so-called multipath problem occurs in which the marker appears double as the radiated radio waves are reflected by structures and the like in the vehicle interior.
  • the information processing device 1 of embodiment 1 is able to use a moving object map as a moving object map obtained without any installation error of the radio wave sensor 21, without being affected by the difficulty of the radio wave sensor 21 in distinguishing between markers and structures in the vehicle cabin other than the markers, or the occurrence of the so-called multipath problem in the radio wave sensor 21.
  • the information processing device 1 is provided outside the detection sensor 2, but this is merely an example.
  • the information processing device 1 may also be provided in the detection sensor 2.
  • the information processing device 1 is an in-vehicle device mounted on a vehicle, and the moving object distribution information acquisition unit 101, the image acquisition unit 102, the camera error detection unit 103, the radio wave sensor error calculation unit 104, the correction unit 105, the output unit 106, and a control unit (not shown) are provided in the in-vehicle device.
  • the system may be configured with an in-vehicle device and a server, with some of the moving object distribution information acquisition unit 101, image acquisition unit 102, camera error detection unit 103, radio wave sensor error calculation unit 104, correction unit 105, output unit 106, and a control unit (not shown) being mounted on the in-vehicle device of the vehicle, and the others being provided in a server connected to the in-vehicle device via a network.
  • the moving object distribution information acquisition unit 101, the image acquisition unit 102, the camera error detection unit 103, the radio wave sensor error calculation unit 104, the correction unit 105, the output unit 106, and a control unit may all be provided in the server.
  • the detection sensor 2 when configured as shown in Fig. 2, it is configured so that at least two of the x-axis, y-axis, and z-axis specific to the camera 22 and the x-axis, y-axis, and z-axis specific to the radio wave sensor 21 coincide with each other when the detection sensor 2 is installed in the vehicle cabin.
  • this is merely an example, and it is not essential that at least two of the x-axis, y-axis, and z-axis specific to the camera 22 and the x-axis, y-axis, and z-axis specific to the radio wave sensor 21 coincide with each other in the detection sensor 2.
  • the information processing device 1 can easily correct the moving object map compared to a configuration in which only one or none of the x-axis, y-axis, and z-axis specific to the camera 22 and the x-axis, y-axis, and z-axis specific to the radio wave sensor 21 coincide.
  • the camera 22 is configured to be provided between the transmitting antenna 11 and the receiving antenna 12 on the common surface of the substrate 24.
  • this is merely an example, and it is not essential that the camera 22 is provided between the transmitting antenna 11 and the receiving antenna 12 on the common surface of the substrate 24 in the detection sensor 2.
  • the information processing device 1 can easily correct the moving object map compared to a configuration in which the camera 22 is not provided between the transmitting antenna 11 and the receiving antenna 12 on the common surface of the substrate 24.
  • FIG. 13A and 13B are diagrams illustrating an example of a hardware configuration of the information processing device 1 according to the first embodiment.
  • the functions of the moving object distribution information acquisition unit 101, the image acquisition unit 102, the camera error detection unit 103, the radio wave sensor error calculation unit 104, the correction unit 105, the output unit 106, and a control unit are realized by the processing circuit 1001. That is, the information processing device 1 includes the processing circuit 1001 for performing control to provide error consideration information for using the moving object map information generated by the radio wave sensor 21 as information obtained in a state where there is no installation error of the radio wave sensor 21.
  • the processing circuitry 1001 may be dedicated hardware as shown in FIG. 13A, or may be a processor 1004 that executes a program stored in memory as shown in FIG. 13B.
  • the processing circuit 1001 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a combination of these.
  • ASIC Application Specific Integrated Circuit
  • FPGA Field-Programmable Gate Array
  • the functions of the moving object distribution information acquisition unit 101, the image acquisition unit 102, the camera error detection unit 103, the radio wave sensor error calculation unit 104, the correction unit 105, the output unit 106, and the control unit are realized by software, firmware, or a combination of software and firmware.
  • the software or firmware is described as a program and stored in the memory 1005.
  • the processor 1004 reads out and executes the program stored in the memory 1005 to execute the functions of the moving object distribution information acquisition unit 101, the image acquisition unit 102, the camera error detection unit 103, the radio wave sensor error calculation unit 104, the correction unit 105, the output unit 106, and the control unit (not shown).
  • the information processing device 1 includes a memory 1005 for storing a program that, when executed by the processor 1004, results in the execution of steps ST1 to ST6 of FIG. 11 described above.
  • the program stored in memory 1005 can also be said to cause the computer to execute the processing procedures or methods of the moving object distribution information acquisition unit 101, the image acquisition unit 102, the camera error detection unit 103, the radio wave sensor error calculation unit 104, the correction unit 105, the output unit 106, and a control unit not shown.
  • memory 1005 includes, for example, non-volatile or volatile semiconductor memory such as RAM, ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), magnetic disk, flexible disk, optical disk, compact disk, mini disk, DVD (Digital Versatile Disc), etc.
  • non-volatile or volatile semiconductor memory such as RAM, ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), magnetic disk, flexible disk, optical disk, compact disk, mini disk, DVD (Digital Versatile Disc), etc.
  • the functions of the moving object distribution information acquisition unit 101, the image acquisition unit 102, the camera error detection unit 103, the radio wave sensor error calculation unit 104, the correction unit 105, the output unit 106, and the control unit may be partially realized by dedicated hardware and partially realized by software or firmware.
  • the functions of the moving object distribution information acquisition unit 101 and the image acquisition unit 102 may be realized by a processing circuit 1001 as dedicated hardware, and the functions of the camera error detection unit 103, the radio wave sensor error calculation unit 104, the correction unit 105, the output unit 106, and the control unit (not shown) may be realized by the processor 1004 reading and executing a program stored in the memory 1005.
  • the information processing device 1 also includes an input interface device 1002 and an output interface device 1003 that perform wired or wireless communication with devices such as a detection sensor 2 or a monitoring device 3 .
  • the information processing device 1 is configured to include an image acquisition unit 102 that acquires an image captured by the camera 22 from a detection sensor 2 having a camera 22 that is provided in the vehicle cabin and captures an image of a target area in the vehicle cabin where at least an occupant may be present, and a radio wave sensor 21 that generates moving object distribution information (moving object map) that three-dimensionally represents the distribution of an area in the vehicle cabin where moving objects are present, based on a reflected wave of radio waves radiated toward the target area in the vehicle cabin and reflected by an object in the vehicle cabin, the image acquisition unit 102 that acquires an image captured by the camera 22, a camera error detection unit 103 that detects an installation error of the camera 22 based on the image acquired by the image acquisition unit 102, a radio wave sensor error calculation unit 104 that calculates an installation error of the radio wave sensor 21 based on the installation error of the camera 22 detected by the camera error detection unit 103, and an output unit 106 that outputs error consideration information for
  • the information processing device 1 can provide error-taking information for using information generated based on the reflected waves received by the radio wave sensor 21, which is installed in the vehicle cabin and can detect the position and movement of moving objects within the vehicle cabin by emitting radio waves into the vehicle cabin and receiving reflected waves from objects of the radiated radio waves, i.e., moving object distribution information, as information obtained without any installation error of the radio wave sensor 21.
  • the information processing device 1 includes a moving object distribution information acquisition unit 101 that acquires moving object distribution information from the detection sensor 2, and a correction unit 105 that corrects the moving object distribution information acquired by the moving object distribution information acquisition unit 101 based on the installation error of the radio wave sensor 21 calculated by the radio wave sensor error calculation unit 104, and the output unit 106 is configured to output the moving object distribution information corrected by the correction unit 105 (corrected moving object map) as error consideration information.
  • the information processing device 1 can provide error-taking information for using information generated based on the reflected waves received by the radio wave sensor 21, which is installed in the vehicle cabin and can detect the position and movement of moving objects within the vehicle cabin by emitting radio waves into the vehicle cabin and receiving reflected waves of the radiated radio waves reflected by an object, i.e., moving object distribution information, as information obtained without any installation error of the radio wave sensor 21.
  • the detection sensor 2 is provided in a vehicle cabin and includes a fixed part 23, a camera 22 provided on the fixed part 23 and configured to capture an image of a target area in the vehicle cabin where at least an occupant may be present, a radio wave sensor 21 provided on the fixed part 23 and configured to generate moving object distribution information that three-dimensionally represents a distribution of an area in the vehicle cabin where moving objects are present, based on a radio wave radiated toward the target area in the vehicle cabin and reflected by an object in the vehicle cabin, an image acquisition unit 102 that acquires the captured image captured by the camera 22, and a camera error detection unit 103 that detects an installation error of the camera 22, based on the captured image acquired by the image acquisition unit 102.
  • the system is equipped with a radio wave sensor error calculation unit 104 that calculates the installation error of the radio wave sensor 21 based on the installation error of the camera 22 detected by the camera error detection unit 103, and an output unit 106 that outputs error consideration information to convert the moving object distribution information generated by the radio wave sensor 21 into moving object distribution information that takes into account the installation error of the radio wave sensor 21 calculated by the radio wave sensor error calculation unit 104, and can be configured such that when installed in the vehicle cabin, at least two of the three-dimensional coordinate axes of the camera 22 (x-axis, y-axis, and z-axis specific to the camera 22) and the three-dimensional coordinate axes of the radio wave sensor 21 (x-axis, y-axis, and z-axis specific to the radio wave sensor 21) coincide with each other. Therefore, the detection sensor 2 can provide error consideration information for using the information generated by the radio wave sensor 21, i.e., the moving object distribution information, as information obtained without any installation error of the radio wave sensor 21.
  • the detection sensor 2 is a camera 22 that is provided in a vehicle interior and captures an image of a target area in the vehicle interior where at least an occupant may be present, and a radio wave sensor 21 that generates moving object distribution information that represents a three-dimensional distribution of an area in the vehicle interior where moving objects are present, based on a radio wave radiated toward the target area in the vehicle interior and reflected by an object in the vehicle interior.
  • the radio wave sensor 21 has a transmitting antenna 11 that radiates radio waves and a receiving antenna 12 that receives reflected waves of the radio waves radiated from the transmitting antenna 11, and the camera 22, the transmitting antenna 11, and the receiving antenna 12.
  • the present invention can be configured to include a substrate 24 on which the cameras 22 and 2 are provided on a common surface, an image acquisition unit 102 that acquires an image captured by the camera 22, a camera error detection unit 103 that detects an installation error of the camera 22 based on the image acquired by the image acquisition unit 102, a radio wave sensor error calculation unit 104 that calculates an installation error of the radio wave sensor 21 based on the installation error of the camera 22 detected by the camera error detection unit 103, and an output unit 106 that outputs error consideration information to convert the moving object distribution information generated by the radio wave sensor 21 into moving object distribution information that takes into account the installation error of the radio wave sensor 21 calculated by the radio wave sensor error calculation unit 104. Therefore, the detection sensor 2 can provide error consideration information for using the information generated by the radio wave sensor 21, i.e., the moving object distribution information, as information obtained without any installation error of the radio wave sensor 21.
  • the information processing system 4 includes a detection sensor 2 having a camera 22 that is provided in a vehicle interior and captures an image of a target area in the vehicle interior where at least an occupant may be present, and a radio wave sensor 21 that generates moving object distribution information (moving object map) that three-dimensionally represents a distribution of an area in the vehicle interior where moving objects are present, based on a radio wave radiated toward the target area in the vehicle interior and reflected by an object in the vehicle interior, and an image acquisition unit 102 that acquires an image captured by the camera 22 from the detection sensor 2, a camera error detection unit 103 that detects an installation error of the camera 22 based on the image acquired by the image acquisition unit 102, and a radio wave sensor error detection unit 103 that calculates an installation error of the radio wave sensor 21 based on the installation error of the camera 22 detected by the camera error detection unit 103.
  • a detection sensor 2 having a camera 22 that is provided in a vehicle interior and captures an image of a target area in the vehicle interior where at least an
  • the information processing device 1 has a calculation unit 104 and an output unit 106 that outputs error consideration information for converting the moving object distribution information generated by the radio wave sensor 21 into moving object distribution information that takes into account the installation error of the radio wave sensor 21 calculated by the radio wave sensor error calculation unit 104, and a detection sensor 2 that is provided in a vehicle cabin and has a fixed part 23, a camera 22 provided on the fixed part 23, and a radio wave sensor 21 provided on the fixed part 23, and is characterized in that, when installed in the vehicle cabin, at least two of the three-dimensional coordinate axes of the camera 22 (x-axis, y-axis, and z-axis specific to the camera 22) and the three-dimensional coordinate axes of the radio wave sensor 21 (x-axis, y-axis, and z-axis specific to the radio wave sensor 21) are consistent.
  • the information processing system 4 can provide error-taking information for using information generated based on the reflected waves received by the radio wave sensor 21, which is installed in the vehicle cabin and can detect the position and movement of moving objects within the vehicle cabin by emitting radio waves into the vehicle cabin and receiving reflected waves of the radiated radio waves reflected by an object, i.e., moving object distribution information, as information obtained without any installation error of the radio wave sensor 21.
  • the information processing system 4 includes a camera 22 that is provided in a vehicle interior and captures an image of a target area in the vehicle interior where at least an occupant may be present, and a radio wave sensor 21 that generates moving object distribution information (moving object map) that three-dimensionally represents a distribution of an area in the vehicle interior where moving objects are present, based on a radio wave radiated toward the target area in the vehicle interior and reflected by an object in the vehicle interior, and the information processing system 4 includes an image acquisition unit 102 that acquires an image captured by the camera 22 from the detection sensor 2, a camera error detection unit 103 that detects an installation error of the camera 22 based on the image acquired by the image acquisition unit 102, and a detection unit 104 that detects the installation error of the camera 22 detected by the camera error detection unit 103.
  • a camera 22 that is provided in a vehicle interior and captures an image of a target area in the vehicle interior where at least an occupant may be present
  • a radio wave sensor 21 that generates moving object distribution information (mov
  • a detection sensor 2 that is provided in a vehicle cabin and has a camera 22, a transmitting antenna 11 that radiates radio waves, and a receiving antenna 12 that receives reflected waves of the radio waves radiated from the transmitting antenna 11, and a substrate 24 on which the camera 22, the transmitting antenna 11, and the receiving antenna 12 are provided on a common surface.
  • the information processing system 4 can provide error-taking information for using information generated based on the reflected waves received by the radio wave sensor 21, which is installed in the vehicle cabin and can detect the position and movement of moving objects within the vehicle cabin by emitting radio waves into the vehicle cabin and receiving reflected waves of the radiated radio waves reflected by an object, i.e., moving object distribution information, as information obtained without any installation error of the radio wave sensor 21.
  • Embodiment 2 the information processing device has a function of correcting the moving object map, and outputs the corrected moving object map as error consideration information.
  • the information processing device does not have a function of correcting the moving object map will be described.
  • FIG. 14 is a diagram showing an example of a configuration of an information processing device 1a according to the second embodiment.
  • the information processing device 1a is connected to a detection sensor 2 and a monitoring device 3.
  • the information processing device 1a and the detection sensor 2 constitute an information processing system 4a.
  • the information processing device 1a, the detection sensor 2, and the monitoring device 3 are mounted on, for example, a vehicle (not shown).
  • the detection sensor 2 according to the second embodiment has a similar configuration to the detection sensor 2 according to the first embodiment described with reference to FIGS. 2 and 3.
  • the monitoring device 3 includes an occupant detection device 31 and a physical build determination device 32, similar to the monitoring device 3 described in the first embodiment. 14, components similar to those of the information processing device 1 described in the first embodiment with reference to FIG. 1 are designated by the same reference numerals and will not be described again.
  • the information processing device 1a according to the second embodiment differs from the information processing device 1 according to the first embodiment in that the information processing device 1a according to the second embodiment does not include the correction unit 105.
  • the specific operation of the output unit 106a is different from the specific operation of the output unit 106 in the information processing device 1 according to the first embodiment.
  • the output unit 106a outputs the moving object distribution information acquired by the moving object distribution information acquisition unit 101 and the sensor installation error information indicating the installation error of the radio wave sensor 21 calculated by the radio wave sensor error calculation unit 104 to the monitoring device 3 as error consideration information. That is, the output unit 106a associates the moving object distribution information output from the moving object distribution information acquisition unit 101 with the sensor installation error information output from the radio wave sensor error calculation unit 104, and outputs it to the monitoring device 3 as error consideration information.
  • the moving object distribution information acquisition unit 101 outputs the acquired moving object distribution information to the output unit 106a.
  • the radio wave sensor error calculation section 104 outputs the sensor installation error information to the output section 106a.
  • the information processing device 1a does not have a function of correcting the moving object map. Therefore, the moving object map included in the error consideration information output by the information processing device 1a to the monitoring device 3 may be a moving object map generated by the radio wave sensor 21 in a state where the radio wave sensor 21 has an installation error. Therefore, in embodiment 2, when monitoring an occupant, the monitoring device 3 performs processing (hereinafter referred to as ⁇ monitoring correction processing'') based on the sensor installation error information contained in the error consideration information so that the moving body map can be used as a moving body map generated without any installation error of the radio wave sensor 21, and then monitors the occupant.
  • ⁇ monitoring correction processing'' processing
  • An example of the monitoring correction process is a process of correcting a region for determining physical build.
  • the occupant detection device 31 and the physique determination device 32 correct the position or orientation of the physique determination region in accordance with the installation error of the radio wave sensor 21 based on the sensor installation error information.
  • FIG. 15 is a flowchart for explaining the operation of the information processing device 1a according to the second embodiment.
  • the information processing device 1a receives an operation instruction
  • the information processing device 1a performs an operation as shown in the flowchart of Fig. 15.
  • a control unit (not shown) of the information processing device 1a receives the operation instruction, and when the control unit receives the operation instruction, the control unit operates each component included in the information processing device 1a.
  • the operation instruction is input, for example, by an administrator etc.
  • the administrator etc. inputs the operation instruction, for example, when installing the detection sensor 2 in the vehicle.
  • step ST11 the moving object distribution information acquisition unit 101 outputs the acquired moving object map to the output unit 106a.
  • step ST14 the radio wave sensor error calculation unit 104 outputs the sensor installation error information to the output unit 106a.
  • the output unit 106a outputs the moving object distribution information acquired by the moving object distribution information acquisition unit 101 in step ST11 and the sensor installation error information indicating the installation error of the radio wave sensor 21 calculated by the radio wave sensor error calculation unit 104 in step ST14 as error consideration information to the monitoring device 3 (step ST15).
  • step ST11 is performed before the processing of step ST12, but this is merely one example.
  • the processing of step ST11 and the processing of step ST12 may be performed in parallel. It is sufficient that the processing of step ST11 is performed before the processing of step ST15 is performed.
  • the information processing device 1a acquires an image captured by the camera 22 from the detection sensor 2, and detects an installation error of the camera 22 based on the captured image.
  • the information processing device 1a detects an installation error of the camera 22, it calculates an installation error of the radio wave sensor 21 based on the installation error of the camera 22.
  • the information processing device 1a outputs, to the monitoring device 3, as error consideration information, the moving object distribution information generated by the radio wave sensor 21 acquired from the detection sensor 2, i.e., the moving object map, and sensor installation error information indicating the calculated installation error of the radio wave sensor 21.
  • the information processing device 1a can provide error consideration information, specifically, in this case, a moving object map and sensor installation error information, for using the moving object map information generated by the radio wave sensor 21 as information obtained in a state where there was no installation error of the radio wave sensor 21.
  • the monitoring device 3 performs monitoring correction processing based on the moving object map and sensor installation error information provided by the information processing device 1a so that the moving object map can be used as a moving object map generated without the installation error of the radio wave sensor 21, and then monitors the occupants.
  • the monitoring device 3 can prevent erroneous monitoring of occupants (for example, erroneous detection of a child being left behind or erroneous determination of the physique of an occupant).
  • the information processing device 1a is provided outside the detection sensor 2, but this is merely an example.
  • the information processing device 1a may also be provided in the detection sensor 2.
  • the information processing device 1a is an in-vehicle device mounted on a vehicle, and the moving object distribution information acquisition unit 101, the image acquisition unit 102, the camera error detection unit 103, the radio wave sensor error calculation unit 104, the output unit 106a, and a control unit (not shown) are provided in the in-vehicle device.
  • the system may be configured with an in-vehicle device and a server, with some of the moving object distribution information acquisition unit 101, image acquisition unit 102, camera error detection unit 103, radio wave sensor error calculation unit 104, output unit 106a, and a control unit (not shown) being mounted on the in-vehicle device of the vehicle, and the others being provided in a server connected to the in-vehicle device via a network. Furthermore, the moving object distribution information acquisition unit 101, the image acquisition unit 102, the camera error detection unit 103, the radio wave sensor error calculation unit 104, the output unit 106a, and a control unit (not shown) may all be provided in the server.
  • the hardware configuration of the information processing device 1a according to the second embodiment is similar to the hardware configuration of the information processing device 1 described in the first embodiment with reference to FIGS. 13A and 13B, and therefore is not illustrated.
  • the functions of the moving object distribution information acquisition unit 101, the image acquisition unit 102, the camera error detection unit 103, the radio wave sensor error calculation unit 104, the output unit 106a, and a control unit are realized by a processing circuit 1001. That is, the information processing device 1a includes the processing circuit 1001 for performing control to provide error consideration information for using the moving object map information generated by the radio wave sensor 21 as information obtained in a state where there is no installation error of the radio wave sensor 21.
  • the processing circuitry 1001 may be dedicated hardware as shown in FIG. 13A, or may be a processor 1004 executing a program stored in a memory 1005 as shown in FIG. 13B.
  • the processing circuit 1001 reads out and executes the programs stored in the memory 1005, thereby executing the functions of the moving object distribution information acquisition unit 101, the image acquisition unit 102, the camera error detection unit 103, the radio wave sensor error calculation unit 104, the output unit 106a, and a control unit (not shown). That is, the information processing device 1a includes a memory 1005 for storing a program that, when executed by the processing circuit 1001, results in the execution of steps ST11 to ST15 in FIG. 15.
  • the information processing device 1a includes an input interface device 1002 and an output interface device 1003 that perform wired or wireless communication with devices such as a detection sensor 2 or a monitoring device 3.
  • the information processing device 1a is configured to include an image acquisition unit 102 that acquires an image captured by the camera 22 from a detection sensor 2 having the camera 22 that is provided in the vehicle cabin and captures an image of a target area in which at least an occupant may be present within the vehicle cabin, and the radio wave sensor 21 that generates moving object distribution information (moving object map) that three-dimensionally represents the distribution of an area in which moving objects are present within the vehicle cabin based on a reflected wave of a radio wave radiated toward the target area within the vehicle cabin and reflected by an object within the vehicle cabin, the image acquisition unit 102 that acquires an image captured by the camera 22, a camera error detection unit 103 that detects an installation error of the camera 22 based on the image acquired by the image acquisition unit 102, a radio wave sensor error calculation unit 104 that calculates an installation error of the radio wave sensor 21 based on the installation error of the camera 22 detected by the camera error detection unit 103, and an output unit 106a that outputs error consideration
  • an image acquisition unit 102 that
  • the information processing device 1a can provide error-taking information for using information generated based on the reflected waves received by the radio wave sensor 21, which is installed in the vehicle cabin and can detect the position and movement of moving objects within the vehicle cabin by emitting radio waves into the vehicle cabin and receiving reflected waves of the radiated radio waves reflected by an object, i.e., moving object distribution information, as information obtained without any installation error of the radio wave sensor 21.
  • the information processing device 1a is equipped with a moving object distribution information acquisition unit 101 that acquires moving object distribution information from the detection sensor 2, and the output unit 106a outputs the moving object distribution information acquired by the moving object distribution information acquisition unit 101 and sensor installation error information indicating the installation error of the radio wave sensor 21 calculated by the radio wave sensor error calculation unit 104 as error consideration information.
  • the information processing device 1a can provide error-taking information for using information generated based on the reflected waves received by the radio wave sensor 21, which is installed in the vehicle cabin and can detect the position and movement of moving objects within the vehicle cabin by emitting radio waves into the vehicle cabin and receiving reflected waves of the radiated radio waves reflected by an object, i.e., moving object distribution information, as information obtained without any installation error of the radio wave sensor 21.
  • the information processing system 4a includes a detection sensor 2 having a camera 22 that is provided in a vehicle interior and captures an image of a target area in the vehicle interior where at least an occupant may be present, and a radio wave sensor 21 that generates moving object distribution information (moving object map) that three-dimensionally represents a distribution of an area in the vehicle interior where moving objects are present, based on a radio wave radiated toward the target area in the vehicle interior and reflected by an object in the vehicle interior, and an image acquisition unit 102 that acquires an image captured by the camera 22 from the detection sensor 2, a camera error detection unit 103 that detects an installation error of the camera 22 based on the image acquired by the image acquisition unit 102, and a radio wave sensor error detection unit 103 that calculates an installation error of the radio wave sensor 21 based on the installation error of the camera 22 detected by the camera error detection unit 103.
  • a detection sensor 2 having a camera 22 that is provided in a vehicle interior and captures an image of a target area in the vehicle interior where at least
  • the information processing device 1a has a calculation unit 104 and an output unit 106a that outputs error consideration information for converting the moving object distribution information generated by the radio wave sensor 21 into moving object distribution information that takes into account the installation error of the radio wave sensor 21 calculated by the radio wave sensor error calculation unit 104, and a detection sensor 2 that is provided in a vehicle cabin and has a fixed part 23, a camera 22 provided on the fixed part 23, and a radio wave sensor 21 provided on the fixed part 23, and is characterized in that, when installed in the vehicle cabin, at least two of the three-dimensional coordinate axes of the camera 22 (x-axis, y-axis, and z-axis specific to the camera 22) and the three-dimensional coordinate axes of the radio wave sensor 21 (x-axis, y-axis, and z-axis specific to the radio wave sensor 21) are consistent.
  • the information processing system 4a can provide error-taking information for using information generated based on the reflected waves received by the radio wave sensor 21, which is installed in the vehicle cabin and can detect the position and movement of moving objects within the vehicle cabin by emitting radio waves into the vehicle cabin and receiving reflected waves of the radiated radio waves reflected by an object, i.e., moving object distribution information, as information obtained without any installation error of the radio wave sensor 21.
  • the information processing system 4a is provided in a vehicle interior and includes a camera 22 that captures an image of a target area in the vehicle interior where at least an occupant may be present, and a radio wave sensor 21 that generates moving object distribution information (moving object map) that three-dimensionally represents a distribution of an area in the vehicle interior where moving objects are present, based on a radio wave radiated toward the target area in the vehicle interior and reflected by an object in the vehicle interior.
  • moving object distribution information moving object map
  • the information processing system 4a includes an image acquisition unit 102 that acquires an image captured by the camera 22 from a detection sensor 2, a camera error detection unit 103 that detects an installation error of the camera 22 based on the image acquired by the image acquisition unit 102, and a detection unit 104 that detects an installation error of the camera 22 based on the installation error of the camera 22 detected by the camera error detection unit 103.
  • the information processing device 1a has a radio wave sensor error calculation unit 104 that calculates the installation error of the radio wave sensor 21, and an output unit 106a that outputs error consideration information to convert the moving object distribution information generated by the radio wave sensor 21 into moving object distribution information that takes into account the installation error of the radio wave sensor 21 calculated by the radio wave sensor error calculation unit 104, and a detection sensor 2 that is provided in the vehicle cabin and has a camera 22, a transmitting antenna 11 that radiates radio waves, and a receiving antenna 12 that receives reflected waves of the radio waves radiated from the transmitting antenna 11, and a substrate 24 on which the camera 22, the transmitting antenna 11, and the receiving antenna 12 are provided on a common surface.
  • the information processing system 4a can provide error-taking information for using information generated based on the reflected waves received by the radio wave sensor 21, which is installed in the vehicle cabin and can detect the position and movement of moving objects within the vehicle cabin by emitting radio waves into the vehicle cabin and receiving reflected waves of the radiated radio waves reflected by an object, i.e., moving object distribution information, as information obtained without any installation error of the radio wave sensor 21.
  • Embodiment 3 In the first embodiment, the information processing device has a function of correcting the moving object map, and outputs the corrected moving object map as error consideration information. In the third embodiment, another embodiment will be described in which the information processing device does not have the function of correcting the moving object map.
  • FIG. 16 is a diagram showing an example of a configuration of an information processing device 1b according to the third embodiment.
  • an information processing device 1b is connected to a detection sensor 2a and a monitoring device 3.
  • the information processing device 1b and the detection sensor 2a configure an information processing system 4b.
  • the information processing device 1b, the detection sensor 2a, and the monitoring device 3 are mounted on, for example, a vehicle (not shown).
  • the detection sensor 2a includes, in addition to the configuration of the detection sensor 2 according to the first embodiment described in the first embodiment with reference to Fig. 2 and Fig. 3, a control unit 25 that changes the orientation of the radio wave sensor 21, i.e., the direction of radio wave radiation by the radio wave sensor 21, based on instruction information output from the information processing device 1b. Details of the instruction information will be described later.
  • the detection sensor 2a includes, for example, a servo motor (not shown) that changes the orientation of the radio wave sensor 21.
  • the control unit 25 changes the orientation of the radio wave sensor 21 by controlling the servo motor based on instruction information output from the information processing device 1b.
  • the radio wave sensor 21 may be a radio wave sensor 21 that can change the direction of radio wave emission by applying signals with different phases to multiple antennas, such as a phased array antenna.
  • the control unit 25 instructs the radio wave sensor 21 to change the direction of radio wave emission based on instruction information output from the information processing device 1b.
  • the control unit 25 changes the orientation of the radio wave sensor 21 based on the instruction information output from the information processing device 1b, thereby correcting the installation error of the radio wave sensor 21.
  • the moving object map generated by the radio wave sensor 21 without any installation error of the radio wave sensor 21 is output from the detection sensor 2a to the information processing device 1b.
  • the monitoring device 3 includes an occupant detection device 31 and a physical size determination device 32, similar to the monitoring device 3 described in the first embodiment.
  • the information processing device 1b according to the third embodiment is different from the information processing device 1 according to the first embodiment in that the information processing device 1b according to the third embodiment does not include the correction unit 105.
  • the specific operation of the output unit 106b is different from the specific operation of the output unit 106 in the information processing device 1 according to the first embodiment.
  • the output unit 106b outputs, based on the sensor installation error information output from the radio wave sensor error calculation unit 104, instruction information to instruct the detection sensor 2a to change the orientation of the radio wave sensor 21 as error consideration information.
  • the instruction information includes information that enables the installation error of the radio wave sensor 21 to be determined.
  • the output unit 106 b outputs the moving object distribution information acquired by the moving object distribution information acquisition unit 101 to the monitoring device 3 .
  • the moving object distribution information acquisition unit 101 outputs the acquired moving object distribution information to the output unit 106b.
  • the radio wave sensor error calculation unit 104 outputs the sensor installation error information to the output unit 106b.
  • the information processing device 1b does not have a function of correcting the moving object map, but instead informs the detection sensor 2a of how much the installation position or the orientation of the radio wave sensor 21 should be changed so that the radio wave sensor 21 is in a state free of installation errors, and outputs instruction information for putting the radio wave sensor 21 in a state free of installation errors.
  • This allows the information processing device 1b to acquire the moving object map generated by the radio wave sensor 21 as a moving object map obtained without any installation error of the radio wave sensor 21.
  • the moving object map output by the information processing device 1b to the monitoring device 3 is also a moving object map obtained without any installation error of the radio wave sensor 21. Therefore, the monitoring device 3 can prevent erroneous monitoring of occupants (for example, erroneous detection of a child being left behind or erroneous determination of the occupant's physical size).
  • FIG. 17 is a flowchart for explaining the operation of the information processing device 1b according to the third embodiment.
  • the information processing device 1b receives an operation instruction
  • the information processing device 1b performs an operation as shown in the flowchart of Fig. 17.
  • a control unit (not shown) of the information processing device 1b receives the operation instruction, and when the control unit receives the operation instruction, the control unit operates each component included in the information processing device 1b.
  • the operation instruction is input, for example, by an administrator, etc.
  • the administrator, etc. inputs the operation instruction, for example, when the detection sensor 2a is installed in the vehicle.
  • step ST111 the moving object distribution information acquisition section 101 outputs the acquired moving object map to the output section 106b.
  • step ST114 the radio wave sensor error calculation section 104 outputs the sensor installation error information to the output section 106b.
  • the output unit 106b outputs instruction information instructing the detection sensor 2a to change the orientation of the radio wave sensor 21 as error consideration information based on sensor installation error information indicating the installation error of the radio wave sensor 21 calculated by the radio wave sensor error calculation unit 104 in step ST114 (step ST115-1). Furthermore, the output unit 106b outputs the moving object distribution information acquired by the moving object distribution information acquisition unit 101 in step ST111, that is, the moving object map, to the monitoring device 3 (step ST115-2).
  • step ST111 is performed before the process of step ST112, but this is merely an example.
  • the process of step ST111 and the process of step ST112 may be performed in parallel. It is sufficient that the process of step ST111 is performed before the process of step ST115-2 is performed.
  • the process of step ST115-1 and the process of step ST115-2 are performed in parallel, but this is merely an example.
  • the process of step ST115-1 and the process of step ST115-2 may be performed before or after each other.
  • the output unit 106b may be configured to output instruction information to the detection sensor 2a, and then output the moving object map generated after the installation error of the radio wave sensor 21 is corrected by the detection sensor 2a, and then output the moving object map to the monitoring device 3.
  • the control unit 25 outputs completion information indicating that the correction has been completed to the output unit 106b of the information processing device 1b.
  • the output unit 106b suspends output of the moving object map to the monitoring device 3 until it acquires completion information from the detection sensor 2a, and starts outputting the moving object map to the monitoring device 3 when it acquires the completion information from the detection sensor 2a.
  • This allows the information processing device 1b to further reduce the occurrence of erroneous monitoring of occupants (for example, erroneous detection of a child being left behind or erroneous determination of the physique of an occupant) by the monitoring device 3.
  • the information processing device 1b acquires from the detection sensor 2a an image captured by the camera 22, and detects, based on the captured image, an installation error of the camera 22.
  • the information processing device 1a detects an installation error of the camera 22, it calculates an installation error of the radio wave sensor 21 based on the installation error of the camera 22.
  • the information processing device 1b Based on the calculated installation error of the radio wave sensor 21, the information processing device 1b outputs, as error consideration information, instruction information that instructs the detection sensor 2a to change the direction of radio wave emission by the radio wave sensor 21.
  • the information processing device 1b can provide error consideration information for using the moving object map information generated by the radio wave sensor 21 as information obtained without any installation error of the radio wave sensor 21, specifically, in this case, instruction information for instructing a change in the orientation of the radio wave sensor 21.
  • the monitoring device 3 is able to monitor the occupants using the moving object map provided by the information processing device 1b and generated in a state free of installation errors of the radio wave sensor 21. As a result, the monitoring device 3 can prevent erroneous monitoring of occupants (for example, erroneous detection of a child being left behind or erroneous determination of the physique of an occupant).
  • the information processing device 1b is provided outside the detection sensor 2a, but this is merely an example.
  • the information processing device 1b may be provided in the detection sensor 2a.
  • the information processing device 1b is an in-vehicle device mounted on a vehicle, and the moving object distribution information acquisition unit 101, the image acquisition unit 102, the camera error detection unit 103, the radio wave sensor error calculation unit 104, the output unit 106b, and a control unit (not shown) are provided in the in-vehicle device.
  • the system may be configured with an in-vehicle device and a server, with some of the moving object distribution information acquisition unit 101, image acquisition unit 102, camera error detection unit 103, radio wave sensor error calculation unit 104, output unit 106b, and a control unit (not shown) being mounted on the in-vehicle device of the vehicle, and the others being provided in a server connected to the in-vehicle device via a network. Furthermore, the moving object distribution information acquisition unit 101, the image acquisition unit 102, the camera error detection unit 103, the radio wave sensor error calculation unit 104, the output unit 106b, and a control unit (not shown) may all be provided in the server.
  • the monitoring device 3 acquires the moving object map generated by the radio wave sensor 21 via the information processing device 1b, but this is merely an example.
  • the monitoring device 3 may acquire the moving object map directly from the detection sensor 2a.
  • the output unit 106b does not output the moving object map to the monitoring device 3.
  • the information processing device 1b may be configured without including the moving object distribution information acquisition unit 101. If the information processing device 1b is configured not to include the moving object distribution information acquisition unit 101, the processing of step ST111 and the processing of step ST115-2 can be omitted in the operation of the information processing device 1b described using the flowchart of Figure 17.
  • the hardware configuration of the information processing device 1b according to the third embodiment is similar to the hardware configuration of the information processing device 1 described in the first embodiment with reference to FIGS. 13A and 13B, and therefore is not illustrated.
  • the functions of the moving object distribution information acquisition unit 101, the image acquisition unit 102, the camera error detection unit 103, the radio wave sensor error calculation unit 104, the output unit 106b, and a control unit are realized by a processing circuit 1001. That is, the information processing device 1b includes the processing circuit 1001 for performing control to provide error consideration information for using the moving object map information generated by the radio wave sensor 21 as information obtained in a state where there is no installation error of the radio wave sensor 21.
  • the processing circuitry 1001 may be dedicated hardware as shown in FIG. 13A, or may be a processor 1004 executing a program stored in a memory 1005 as shown in FIG. 13B.
  • the processing circuit 1001 reads out and executes the programs stored in the memory 1005, thereby executing the functions of the moving object distribution information acquisition unit 101, the image acquisition unit 102, the camera error detection unit 103, the radio wave sensor error calculation unit 104, the output unit 106b, and a control unit (not shown). That is, the information processing device 1a includes a memory 1005 for storing a program that, when executed by the processing circuit 1001, results in the execution of steps ST111 to ST115-1 and ST115-2 in FIG. 17 described above.
  • the program stored in the memory 1005 causes the computer to execute the procedures or methods of the processing of the moving object distribution information acquisition unit 101, the image acquisition unit 102, the camera error detection unit 103, the radio wave sensor error calculation unit 104, the output unit 106b, and a control unit (not shown).
  • the information processing device 1b includes an input interface device 1002 and an output interface device 1003 that perform wired or wireless communication with devices such as the detection sensor 2a or the monitoring device 3.
  • the information processing device 1b is configured to include an image acquisition unit 102 that acquires an image captured by the camera 22 from a detection sensor 2a having a camera 22 that is provided in the vehicle cabin and captures an image of a target area in which at least an occupant may be present in the vehicle cabin, and a radio wave sensor 21 that generates moving object distribution information (moving object map) that three-dimensionally represents the distribution of an area in which moving objects are present in the vehicle cabin based on a reflected wave of radio waves emitted toward the target area in the vehicle cabin and reflected by an object in the vehicle cabin, the image acquisition unit 102 that acquires an image captured by the camera 22, a camera error detection unit 103 that detects an installation error of the camera 22 based on the image acquired by the image acquisition unit 102, a radio wave sensor error calculation unit 104 that calculates an installation error of the radio wave sensor 21 based on the installation error of the camera 22 detected by the camera error detection unit 103, and an output unit 106b that outputs error
  • an image acquisition unit 102 that
  • the information processing device 1b can provide error-taking information for using information generated based on the reflected waves received by the radio wave sensor 21, which is installed in the vehicle cabin and can detect the position and movement of moving objects within the vehicle cabin by emitting radio waves into the vehicle cabin and receiving reflected waves of the radiated radio waves reflected by an object, i.e., moving object distribution information, as information obtained without any installation error of the radio wave sensor 21.
  • the output unit 106b outputs, as error consideration information, instruction information for instructing the detection sensor 2a to change the direction of radio wave radiation by the radio wave sensor 21. Therefore, the information processing device 1b can provide error-taking information for using information generated based on the reflected waves received by the radio wave sensor 21, which is installed in the vehicle cabin and can detect the position and movement of moving objects within the vehicle cabin by emitting radio waves into the vehicle cabin and receiving reflected waves of the radiated radio waves reflected by an object, i.e., moving object distribution information, as information obtained without any installation error of the radio wave sensor 21.
  • the information processing system 4b includes a detection sensor 2a having a camera 22 that is provided in a vehicle interior and captures an image of a target area in the vehicle interior where at least an occupant may be present, and a radio wave sensor 21 that generates moving object distribution information (moving object map) that three-dimensionally represents a distribution of an area in the vehicle interior where moving objects are present, based on a radio wave radiated toward the target area in the vehicle interior and reflected by an object in the vehicle interior, and an image acquisition unit 102 that acquires an image captured by the camera 22, a camera error detection unit 103 that detects an installation error of the camera 22 based on the image acquired by the image acquisition unit 102, and a radio wave sensor 21 that detects an installation error of the camera 22 based on the installation error of the camera 22 detected by the camera error detection unit 103.
  • moving object distribution information moving object map
  • the information processing device 1b has a radio wave sensor error calculation unit 104 that calculates an installation error, and an output unit 106b that outputs error consideration information for converting moving object distribution information generated by the radio wave sensor 21 into moving object distribution information that takes into account the installation error of the radio wave sensor 21 calculated by the radio wave sensor error calculation unit 104, and a detection sensor 2a that is provided in a vehicle cabin and has a fixed part 23, a camera 22 provided on the fixed part 23, and a radio wave sensor 21 provided on the fixed part 23, and is characterized in that when installed in the vehicle cabin, at least two of the positions of the camera 22 and the radio wave sensor 21 in the width direction, height direction, or depth direction in the three-dimensional space within the vehicle cabin coincide.
  • the information processing system 4b can provide error-taking information for using information generated based on the reflected waves received by the radio wave sensor 21, which is installed in the vehicle cabin and can detect the position and movement of moving objects within the vehicle cabin by emitting radio waves into the vehicle cabin and receiving reflected waves of the radiated radio waves reflected by an object, i.e., moving object distribution information, as information obtained without any installation error of the radio wave sensor 21.
  • the information processing system 4b is provided in a vehicle interior and includes a camera 22 that captures an image of a target area in the vehicle interior where at least an occupant may be present, and a radio wave sensor 21 that generates moving object distribution information (moving object map) that three-dimensionally represents a distribution of an area in the vehicle interior where moving objects are present, based on a radio wave radiated toward the target area in the vehicle interior and reflected by an object in the vehicle interior.
  • moving object distribution information moving object map
  • the information processing system 4b includes an image acquisition unit 102 that acquires an image captured by the camera 22 from the detection sensor 2a, a camera error detection unit 103 that detects an installation error of the camera 22 based on the image acquired by the image acquisition unit 102, and a detection unit 104 that detects an installation error of the camera 22 based on the installation error of the camera 22 detected by the camera error detection unit 103.
  • the information processing device 1b has a radio wave sensor error calculation unit 104 that calculates the installation error of the radio wave sensor 21, and an output unit 106b that outputs error consideration information to convert the moving object distribution information generated by the radio wave sensor 21 into moving object distribution information that takes into account the installation error of the radio wave sensor 21 calculated by the radio wave sensor error calculation unit 104, and a detection sensor 2a that is provided in the vehicle cabin and has a camera 22, a transmitting antenna 11 that radiates radio waves, and a receiving antenna 12 that receives reflected waves of the radio waves radiated from the transmitting antenna 11, and a substrate 24 on which the camera 22, the transmitting antenna 11, and the receiving antenna 12 are provided on a common surface.
  • the information processing system 4b can provide error-taking information for using information generated based on the reflected waves received by the radio wave sensor 21, which is installed in the vehicle cabin and can detect the position and movement of moving objects within the vehicle cabin by emitting radio waves into the vehicle cabin and receiving reflected waves of the radiated radio waves reflected by an object, i.e., moving object distribution information, as information obtained without any installation error of the radio wave sensor 21.
  • the output unit 106b of the information processing device 1b outputs instruction information to the detection sensor 2a instructing the detection sensor 2a to change the direction of radio wave radiation by the radio wave sensor 21, and the detection sensor 2a is configured to have a control unit 25 that changes the direction of radio wave radiation by the radio wave sensor 21 based on the instruction information output from the output unit 106b.
  • the information processing system 4b can provide error-taking information for using information generated based on the reflected waves received by the radio wave sensor 21, which is installed in the vehicle cabin and can detect the position and movement of moving objects within the vehicle cabin by emitting radio waves into the vehicle cabin and receiving reflected waves of the radiated radio waves reflected by an object, i.e., moving object distribution information, as information obtained without any installation error of the radio wave sensor 21.
  • the occupant detection device 31 or the physical size determination device 32 may be made to change the threshold value used when determining the physical size of a moving object, in other words, an occupant, depending on the installation error of the radio wave sensor 21.
  • the information processing devices 1, 1a, 1b include information capable of identifying an installation error of the radio wave sensor 21 in the error consideration information for the occupant detection device 31 and the physique determination device 32.
  • the occupant detection device 31 or the physique determination device 32 identifies an installation error of the radio wave sensor 21 from the information capable of identifying the installation error of the radio wave sensor 21 output from the information processing devices 1, 1a, 1b, and makes a change, such as increasing or decreasing a threshold value used when determining the occupant's physique, in accordance with the installation error. This allows the occupant detection device 31 or the physique determination device 32 to determine the physique of the occupant taking into consideration the fact that the moving body map including errors was originally output from the radio wave sensor 21.
  • the moving body reflection characteristics change, and therefore the occupant detection device 31 or the physique determination device 32 can determine the physique of the occupant taking into account the fact that errors also occur in the information associated with the grids on the moving body map.
  • the monitoring device 3 includes the occupant detection device 31 and the physique determination device 32, but this is merely an example.
  • the monitoring device 3 may include either the occupant detection device 31 or the physique determination device 32.
  • the information processing devices 1 , 1 a , and 1 b can also provide error consideration information to devices other than the monitoring device 3 .
  • any combination of the embodiments may be used, or any component of each embodiment may be modified, or any component of each embodiment may be omitted.
  • the information processing device can provide error-taking information for using information generated based on the reflected waves received by a radio wave sensor that is installed in the vehicle cabin and can detect the position and movement of a moving object in the vehicle cabin by emitting radio waves into the vehicle cabin and receiving the reflected waves of the radiated radio waves reflected by an object, as information obtained without any installation error of the radio wave sensor.
  • 1, 1a, 1b information processing device
  • 101 moving object distribution information acquisition unit
  • 102 image acquisition unit
  • 103 camera error detection unit
  • 104 radio wave sensor error calculation unit
  • 105 correction unit
  • 106, 106a, 106b output unit
  • 2, 2a detection sensor
  • 21 radio wave sensor
  • 22 camera
  • 23 fixed part
  • 24 board
  • 25 control unit
  • 3 monitoring device
  • 32 physical size determination device
  • 4a, 4b information processing system
  • 1001 processing circuit
  • 1002 input interface device
  • 1003 output interface device
  • 1004 processor
  • 1005 memory.

Landscapes

  • Engineering & Computer Science (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Physics & Mathematics (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • General Physics & Mathematics (AREA)
  • Electromagnetism (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Radar Systems Or Details Thereof (AREA)
PCT/JP2022/047262 2022-12-22 2022-12-22 情報処理装置、検知センサ、情報処理システム、および、情報処理方法 WO2024134815A1 (ja)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP2024565490A JPWO2024134815A1 (enrdf_load_stackoverflow) 2022-12-22 2022-12-22
PCT/JP2022/047262 WO2024134815A1 (ja) 2022-12-22 2022-12-22 情報処理装置、検知センサ、情報処理システム、および、情報処理方法

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2022/047262 WO2024134815A1 (ja) 2022-12-22 2022-12-22 情報処理装置、検知センサ、情報処理システム、および、情報処理方法

Publications (1)

Publication Number Publication Date
WO2024134815A1 true WO2024134815A1 (ja) 2024-06-27

Family

ID=91587866

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2022/047262 WO2024134815A1 (ja) 2022-12-22 2022-12-22 情報処理装置、検知センサ、情報処理システム、および、情報処理方法

Country Status (2)

Country Link
JP (1) JPWO2024134815A1 (enrdf_load_stackoverflow)
WO (1) WO2024134815A1 (enrdf_load_stackoverflow)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2007163258A (ja) * 2005-12-13 2007-06-28 Alpine Electronics Inc 車載センサの補正装置および方法
JP2014228943A (ja) * 2013-05-20 2014-12-08 日本電産エレシス株式会社 車両用外界センシング装置、その軸ズレ補正プログラム及びその軸ズレ補正方法
WO2021124548A1 (ja) * 2019-12-20 2021-06-24 三菱電機株式会社 車内監視装置、車内監視システム、及び車内監視方法
WO2022162852A1 (ja) * 2021-01-29 2022-08-04 三菱電機株式会社 電波センサ及び乗員検知装置

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10349032B2 (en) * 2016-09-30 2019-07-09 Veoneer Us, Inc. Vehicle occupant head positioning system

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2007163258A (ja) * 2005-12-13 2007-06-28 Alpine Electronics Inc 車載センサの補正装置および方法
JP2014228943A (ja) * 2013-05-20 2014-12-08 日本電産エレシス株式会社 車両用外界センシング装置、その軸ズレ補正プログラム及びその軸ズレ補正方法
WO2021124548A1 (ja) * 2019-12-20 2021-06-24 三菱電機株式会社 車内監視装置、車内監視システム、及び車内監視方法
WO2022162852A1 (ja) * 2021-01-29 2022-08-04 三菱電機株式会社 電波センサ及び乗員検知装置

Also Published As

Publication number Publication date
JPWO2024134815A1 (enrdf_load_stackoverflow) 2024-06-27

Similar Documents

Publication Publication Date Title
JP7313586B2 (ja) 電波センサ及び乗員検知装置
JP6781265B2 (ja) 非構造化データを使用したレーダ取り付け判定
US10581150B2 (en) Method and apparatus for radar accuracy measurements
JP6371534B2 (ja) レーダ装置、車両制御システム、および、信号処理方法
US8717224B2 (en) Integrated radar apparatus and intergrated antenna apparatus
US10349032B2 (en) Vehicle occupant head positioning system
US20140062759A1 (en) Object detection apparatus and method
US10705195B2 (en) Distance measuring apparatus and distance measuring method
JP2018503071A (ja) レーダセンサ、レーダセンサシステム、および点反射物体と面反射物体を測定するための水平垂直デジタルビーム形成を用いて物体の位置を特定する方法
US11169247B2 (en) Distance measuring apparatus, distance measuring method, and non-transitory computer-readable storage medium for storing distance measuring program
JP2011099683A (ja) 物体検出装置
US12235099B2 (en) Axial deviation estimating device
CN111801718A (zh) 物体探测装置、物体探测方法、以及记录介质
KR20190084730A (ko) 비선형 비행 궤적에서 스퀸트 스포트라이트 모드를 위한 sar 신호 처리기와 항공기 탑재 합성구경 레이더 및 방법
CN112888961A (zh) 电子设备、电子设备的控制方法以及电子设备的控制程序
WO2023127072A1 (ja) 置き去り検知装置及び置き去り検知方法
WO2024134815A1 (ja) 情報処理装置、検知センサ、情報処理システム、および、情報処理方法
JP3324530B2 (ja) レーダ装置
US20230168370A1 (en) Imaging method and apparatus, radar system, electronic device, and storage medium
JP7477400B2 (ja) 軸ずれ推定装置
JP7481196B2 (ja) 軸ずれ推定装置
JP3335832B2 (ja) レーダ受信装置
EP4314881A1 (en) Illumination device and method for time-of-flight cameras
JP7156103B2 (ja) 生体情報検出装置
JP2020169868A (ja) 送受信モジュールおよびレーダ装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22969201

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2024565490

Country of ref document: JP

NENP Non-entry into the national phase

Ref country code: DE