WO2023071992A1 - 多传感器信号融合的方法、装置、电子设备及存储介质 - Google Patents

多传感器信号融合的方法、装置、电子设备及存储介质 Download PDF

Info

Publication number
WO2023071992A1
WO2023071992A1 PCT/CN2022/127075 CN2022127075W WO2023071992A1 WO 2023071992 A1 WO2023071992 A1 WO 2023071992A1 CN 2022127075 W CN2022127075 W CN 2022127075W WO 2023071992 A1 WO2023071992 A1 WO 2023071992A1
Authority
WO
WIPO (PCT)
Prior art keywords
signal data
signal
target
sensor
laser
Prior art date
Application number
PCT/CN2022/127075
Other languages
English (en)
French (fr)
Inventor
邓永强
杨炎龙
李娟娟
吴雷
Original Assignee
北京万集科技股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京万集科技股份有限公司 filed Critical 北京万集科技股份有限公司
Publication of WO2023071992A1 publication Critical patent/WO2023071992A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
    • G01S13/86—Combinations of radar systems with non-radar systems, e.g. sonar, direction finder
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
    • G01S13/87—Combinations of radar systems, e.g. primary radar and secondary radar
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/86—Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
    • G01S7/02—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S13/00
    • G01S7/41—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S13/00 using analysis of echo signal for target characterisation; Target signature; Target cross-section
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
    • G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
    • Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
    • Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation

Definitions

  • the present application relates to the technical field of intelligent monitoring, in particular to a method, device, electronic equipment and storage medium for multi-sensor signal fusion.
  • Detection sensors are widely used in the field of intelligent monitoring technology.
  • Common detection sensors include, but are not limited to, lidar sensors, millimeter-wave radar sensors, and visible light sensors.
  • the working frequency bands of different detection sensors are different, which makes the detection performance of different detection sensors different. For example, the penetration ability of millimeter-wave radar sensors is stronger than that of lidar sensors, but the detection accuracy of lidar sensors is higher than that of millimeter-wave radar sensors.
  • the lidar sensor encounters challenges during detection, for example, the lidar sensor encounters severe weather such as fog and sandstorms, it will cause some fuzzy but real target related data to be filtered out during the screening process. In this way, there will be situations such as missed detection during fusion, which reduces the accuracy of the final fusion data generated.
  • the embodiment of the present application provides a multi-sensor signal fusion method, device, electronic equipment, and storage medium, which can solve the problem of missed detection during fusion in related technologies, which reduces the accuracy of the final generated fusion data. question. Described technical scheme is as follows:
  • a method for multi-sensor signal fusion comprising:
  • the spatial signal description matrix can describe the signal echo quality of the target
  • the fusion processing of the plurality of signal data includes:
  • the evaluation parameter set includes said at least one evaluation parameter
  • Signal fusion processing is performed based on the multiple sets of evaluation parameter sets.
  • the plurality of signal data includes first laser signal data and millimeter wave signal data
  • the echo front position refers to the initial detection of the target by the laser radar sensor
  • the saturation of the laser peak is used to indicate the confidence that the target is a real target.
  • the performing signal fusion processing based on the multiple sets of evaluation parameter sets includes:
  • the millimeter wave peak position Based on the millimeter wave peak position and the millimeter wave echo energy, determine the variance of the millimeter wave peak position to obtain a first variance, and the first variance is used to indicate that the target is at the millimeter wave peak the probability distribution at the location;
  • the variance of the echo front position Based on the echo front position and the laser peak saturation, determine the variance of the echo front position to obtain a second variance, the second variance is used to indicate the position of the target at the echo front position probability distribution;
  • the spatial signal description matrix is determined based on the millimeter wave peak position, the echo front position, the first variance, and the second variance.
  • the determining the spatial signal description matrix based on the millimeter wave peak position, the echo front position, the first variance, and the second variance includes:
  • the spatial signal description matrix is determined through the following formulas (1) to (3):
  • the ⁇ 12 includes (x 12 , y 12 ), the ⁇ 1 is the position of the millimeter wave crest, including (x 1 , y 1 ), and the ⁇ 2 is the position of the echo front, including ( x 2 , y 2 ), the ⁇ 1 is the first variance, the ⁇ 2 is the second variance, the (x, y) is a variable, and the ⁇ 12 is the joint confidence, so
  • the above V is the joint confidence distribution.
  • the output of fusion data based on the spatial signal description matrix includes:
  • the fusion processing of the plurality of signal data includes:
  • Fusion processing is performed based on the plurality of target signal data.
  • the plurality of detection sensors include lidar sensors and millimeter wave radar sensors, and the plurality of signal data includes first laser signal data and millimeter wave signal data;
  • the location association based on the plurality of signal data includes:
  • Position correlation is performed based on the first dynamic target signal data and the second dynamic target signal data.
  • the millimeter wave signal data includes velocity
  • the signal data of the static target detected by the millimeter wave radar sensor is filtered out from the millimeter wave signal data to obtain the second dynamic target signal data
  • the millimeter wave signal data obtained after the deletion processing is determined as the second dynamic target signal data.
  • the filtering out the signal data of the static target detected by the lidar sensor from the first laser signal data to obtain the first dynamic target signal data includes:
  • the second laser signal data is a frame of laser signal data adjacent to the first laser signal data
  • a signal data difference obtained by subtracting the first laser signal data from the second laser signal data is used as the second dynamic target signal data.
  • the feature dimension space includes angle features and distance features
  • the fusion data is output based on the spatial signal description matrix, including:
  • the angle and the distance are determined as the fused data.
  • the method also includes:
  • the separately synchronizing each sensor sampling signal of the plurality of sensor sampling signals to the same feature dimension space includes: respectively synchronizing the regularized sensor sampling signals to the feature dimension space.
  • the method also includes:
  • time synchronization refers to triggering the work of other detection sensors in the plurality of sensors by one detection sensor in the plurality of sensors
  • space synchronization refers to A spatial transformation matrix is determined by multiple detections of the same test target by the plurality of detection sensors.
  • the fusion processing of the plurality of signal data includes:
  • Fusion processing is performed on the plurality of aligned signal data.
  • the multiple detection sensors include lidar sensors and millimeter wave radar sensors
  • the synchronizing each sensor sampling signal in the plurality of sensor sampling signals to the same feature dimension space respectively includes:
  • the feature dimensions included in the first feature dimension space are frame amount features, scanning beam features, and time features, and the feature dimensions included in the space are velocity features, angle features, and distance features;
  • the sampling signal of the millimeter-wave radar sensor is converted from the second feature dimension space to the feature dimension space, and the feature dimensions included in the second feature dimension space are chirp feature, radiation antenna feature, and time feature.
  • a multi-sensor signal fusion device comprising:
  • the obtaining module is used to obtain the sampling signal of each detection sensor in the multiple detection sensors for the same detection range at the same time, and obtain multiple sensor sampling signals, and the working frequency bands of the multiple detection sensors are different;
  • a synchronization module configured to synchronize each sensor sampling signal in the plurality of sensor sampling signals to the same feature dimension space to obtain multiple signal data, wherein the feature dimension space is based on each detection sensor
  • the detection dimension and system output requirements are set;
  • a fusion module configured to perform fusion processing on the plurality of signal data to obtain a spatial signal description matrix, where the spatial signal description matrix can describe the signal echo quality of the target;
  • An output module configured to output fusion data based on the spatial signal description matrix.
  • the fusion module is used for:
  • the evaluation parameter set includes said at least one evaluation parameter
  • Signal fusion processing is performed based on the multiple sets of evaluation parameter sets.
  • the plurality of signal data includes the first laser signal data and millimeter wave signal data; the fusion module is used for:
  • the echo front position refers to the initial detection of the target by the laser radar sensor
  • the saturation of the laser peak is used to indicate the confidence that the target is a real target.
  • the fusion module is used for:
  • the millimeter wave peak position Based on the millimeter wave peak position and the millimeter wave echo energy, determine the variance of the millimeter wave peak position to obtain a first variance, and the first variance is used to indicate that the target is at the millimeter wave peak the probability distribution at the location;
  • the variance of the echo front position Based on the echo front position and the laser peak saturation, determine the variance of the echo front position to obtain a second variance, the second variance is used to indicate the position of the target at the echo front position probability distribution;
  • the spatial signal description matrix is determined based on the millimeter wave peak position, the echo front position, the first variance, and the second variance.
  • the fusion module is used for:
  • the spatial signal description matrix is determined through the following formulas (1) to (3):
  • the ⁇ 12 includes (x 12 , y 12 ), the ⁇ 1 is the position of the millimeter wave crest, including (x 1 , y 1 ), and the ⁇ 2 is the position of the echo front, including ( x 2 , y 2 ), the ⁇ 1 is the first variance, the ⁇ 2 is the second variance, the (x, y) is a variable, and the ⁇ 12 is the joint confidence, so
  • the above V is the joint confidence distribution.
  • the output module is used to:
  • the fusion module is used for:
  • Fusion processing is performed based on the plurality of target signal data.
  • the plurality of detection sensors include lidar sensors and millimeter wave radar sensors, and the plurality of signal data includes first laser signal data and millimeter wave signal data;
  • the fusion module is used to: filter out the signal data of the static target detected by the lidar sensor from the first laser signal data to obtain the first dynamic target signal data, and filter out the signal data from the millimeter wave signal data
  • the signal data of the static target detected by the millimeter-wave radar sensor is used to obtain the second dynamic target signal data;
  • the specific implementation of performing position association by the fusion module based on the plurality of signal data includes: performing position association based on the first dynamic target signal data and the second dynamic target signal data.
  • the millimeter wave signal data includes speed
  • the fusion module is used for:
  • the millimeter wave signal data obtained after the deletion processing is determined as the second dynamic target signal data.
  • the fusion module is used for:
  • the second laser signal data is a frame of laser signal data adjacent to the first laser signal data
  • a signal data difference obtained by subtracting the first laser signal data from the second laser signal data is used as the second dynamic target signal data.
  • the feature dimension space includes angle features and distance features
  • the output module is used for:
  • the angle and the distance are determined as the fused data.
  • the output module is also used for:
  • the synchronization module is also used for:
  • the regularized sensor sampling signals are respectively synchronized to the feature dimension space.
  • the synchronization module is also used for:
  • time synchronization refers to triggering the work of other detection sensors in the plurality of sensors by one detection sensor in the plurality of sensors
  • space synchronization refers to A spatial transformation matrix is determined by multiple detections of the same test target by the plurality of detection sensors.
  • the fusion module is used for:
  • Fusion processing is performed on the plurality of aligned signal data.
  • the multiple detection sensors include lidar sensors and millimeter wave radar sensors
  • the synchronization module is used for:
  • the feature dimensions included in the first feature dimension space are frame amount features, scanning beam features, and time features, and the feature dimensions included in the space are velocity features, angle features, and distance features;
  • the sampling signal of the millimeter-wave radar sensor is converted from the second feature dimension space to the feature dimension space, and the feature dimensions included in the second feature dimension space are chirp feature, radiation antenna feature, and time feature.
  • an electronic device including a memory, a processor, and a computer program stored in the memory and operable on the processor.
  • the processor executes the computer program, the above-mentioned first The method described in any one of the aspects.
  • a computer-readable storage medium where instructions are stored on the computer-readable storage medium, and when the instructions are executed by a processor, the method described in any one of the above-mentioned first aspects is implemented.
  • a computer program product containing instructions, which, when run on a computer, causes the computer to execute the method described in any one of the above first aspects.
  • the sampling signal of each detection sensor in the multiple detection sensors in the same detection range is acquired at the same time, and multiple sensor sampling signals are obtained, and the working frequency bands of the multiple detection sensors are different. Synchronize each of the multiple sensor sampling signals to the same feature dimension space to obtain multiple signal data, wherein the feature dimension space is set according to the detection dimension of each detection sensor and the system output requirements. Multiple signal data are fused to obtain a spatial signal description matrix, which can describe the signal echo quality of the target. Based on the spatial signal description matrix, the fusion data is output. In this way, data fusion based on the original sensor sampling signal of the detection sensor can improve the robustness of detection while avoiding missed detection due to fusion based on point cloud data or target detection results, and improve the final fusion. Data Accuracy.
  • Fig. 1 is a flowchart of a method for multi-sensor signal fusion according to an exemplary embodiment
  • Fig. 2 is a flow chart of a method for multi-sensor signal fusion according to another exemplary embodiment
  • Fig. 3 is a schematic diagram showing a sampling signal according to an exemplary embodiment
  • Fig. 4 is a flowchart of a method for multi-sensor signal fusion according to another exemplary embodiment
  • Fig. 5 is a flow chart of a method for multi-sensor signal fusion according to another exemplary embodiment
  • Fig. 6 is a schematic diagram of a point cloud diagram according to an exemplary embodiment
  • Fig. 7 is a schematic structural diagram of a multi-sensor signal fusion device according to an exemplary embodiment
  • Fig. 8 is a schematic structural diagram of an electronic device according to an exemplary embodiment.
  • Lidar sensor Through laser scanning, it can restore the three-dimensional perception information of the surrounding environment within the detection range. Its working principle is to use TOF (Time of flight, time-of-flight ranging) method for ranging, and perform AD (Analog-to-digital, analog-to-digital) isochronous sampling at the back end of the receiving photodiode, and complete each After the angle is sampled, the sampled signal amplitude sequence is output from the network port to obtain the laser sampled signal.
  • LiDAR sensors can achieve fine 3D perception and can achieve an angular resolution of 0.2 degrees. However, when there is noise visible to the naked eye in the surrounding environment, the detection ability of the lidar sensor will drop sharply.
  • the diameter of the particle polymer in the air is equivalent to the wavelength of the laser radar sensor, and the amount of particles is sufficient, so that the laser scanned by the laser radar sensor Attenuation and backscattering into these particle aggregates occurs.
  • false detection echoes for particle aggregates in the air are generated, and on the other hand, the transmission power of the laser is reduced, thereby reducing the detection range of the lidar sensor.
  • Millimeter wave radar sensor Perceive the surrounding environment by emitting electromagnetic waves. Its working principle is to use modulation methods including FMCW (Frequency Modulated Continuous Wave) for space measurement, and to send and receive continuously modulated electromagnetic waves with a base frequency of 24GHz or 77GHz from the radio frequency antenna. Perform AD sampling on the echo received by the receiving antenna, store it in the register, and output it from the network interface after completing a frame of signal reception to obtain a millimeter wave sampling signal.
  • the millimeter-wave radar sensor has the characteristics of being less affected by noise, that is, it has strong anti-interference ability. However, the detection accuracy of the millimeter-wave radar sensor is low, and it can achieve an angular resolution of 1 degree.
  • Time synchronization including hardware time synchronization and software time synchronization.
  • hardware time synchronization means that one of them sends a synchronization trigger signal to the other at regular intervals, and the other performs a detection operation after receiving the synchronization trigger signal. Both output sampling signals at the same time.
  • the software time synchronization is based on the fact that the two detection sensors have fixed output frame rates respectively, and each frame output by the two sensors is aligned on the same time coordinate axis for frame number alignment and translation alignment.
  • Spatial synchronization set multiple detection sensors with visible markers (such as metal markers) in the bands of each detection sensor, use multiple detection sensors to simultaneously sample the space where the markers are located, and rotate and translate the markers in space align.
  • Multiple sets of sampling data can be obtained by repeating the operations of sampling and rotation and translation.
  • the position information of the markers is calculated separately, and multiple sets of position information are obtained.
  • the rotation matrix and translation vector are determined by solving equations, and the space transformation matrix is obtained.
  • the data of multiple detection sensors can be unified into the same coordinate system by using the determined space transformation matrix.
  • the method for multi-sensor signal fusion may be performed by an electronic device.
  • the electronic device may be configured or connected with multiple detection sensors, and the working frequency bands of the multiple detection sensors are different.
  • the multiple detection sensors include a laser radar sensor and a millimeter wave radar sensor.
  • the lidar sensor may include but not limited to any one of 8-line lidar, 16-line lidar, 24-line lidar, 32-line lidar, 64-line lidar, and 128-line lidar.
  • the millimeter-wave radar sensor may include, but is not limited to, any one of 77GHz millimeter-wave radar and 24GHz millimeter-wave radar.
  • the lidar sensor and the millimeter-wave radar sensor can be installed according to actual needs, for example, they can be fixed on the traffic roadside markers (crossbars or poles) to be detected by the lidar sensor and the millimeter-wave radar sensor respectively.
  • target detection As an example, the targets to be detected may include but not limited to vehicles, pedestrians, non-motorized vehicles, and trees.
  • the millimeter-wave radar sensor can use an antenna array with an area array, and its field of view is a forward 180° space or less. For lidar sensors, this includes a limited forward field of view and a 360° surround view.
  • the forward-scanning lidar sensor cooperates with a millimeter-wave radar sensor with an area array antenna array to complete the forward-scanning Scanning imaging of the field of view.
  • the lidar sensor with a 360° surround-view field of view 3-4 millimeter-wave radar sensors with area array antenna arrays are used as needed, and the fields of view of multiple millimeter-wave radar sensors partially overlap.
  • a lidar sensor with a 360° surround view it is equipped with a millimeter-wave radar sensor that also has a 360° surround view.
  • electronic devices may include, but are not limited to, wearable devices, terminal devices, in-vehicle devices, camera devices, and roadside base stations.
  • wearable devices may include, but not limited to, smart watches, smart bracelets, and smart earmuffs.
  • terminal devices may include, but are not limited to, mobile phones, tablet computers, augmented reality (augmented reality, AR)/virtual reality (virtual reality, VR) equipment, notebook computers, ultra-mobile personal computers (ultra-mobile personal computer, UMPC), Netbooks, personal digital assistants (PDAs).
  • PDAs personal digital assistants
  • the roadside base station is an important infrastructure for intelligent transportation vehicle-road coordination, and a service station integrating perception, computing, and communication capabilities.
  • the roadside base station may also be called a smart base station or a roadside fusion sensing system.
  • FIG. 1 is a flow chart of a method for multi-sensor signal fusion according to an exemplary embodiment.
  • the method may be applied to the above-mentioned electronic device.
  • the method may include the following steps:
  • Step 101 Obtain the sampling signal of each detection sensor in the multiple detection sensors in the same detection range at the same time to obtain multiple sensor sampling signals, and the working frequency bands of the multiple detection sensors are different.
  • the plurality of detection sensors includes lidar sensors and millimeter wave radar sensors. In another embodiment, the plurality of detection sensors includes a millimeter wave radar sensor and a visible light detection sensor. In yet another embodiment, the plurality of detection sensors include millimeter wave radar sensors and infrared detection sensors.
  • an electronic device detects a target through a plurality of configured detection sensors. During the detection process, the electronic device samples and processes the echoes of each detection sensor in the multiple detection sensors to obtain multiple sensor sampling signals.
  • time synchronization and space synchronization can also be performed on the multiple detection sensors.
  • time synchronization refers to triggering the work of other detection sensors among the multiple sensors by one detection sensor among the multiple sensors.
  • time synchronization may include: during the rotation scanning process of the lidar sensor, whenever the motor of the lidar sensor rotates When passing through its own zero point, a synchronous trigger signal is sent to the millimeter wave radar sensor.
  • the millimeter wave radar sensor receives the synchronous trigger signal, it starts to send continuous electromagnetic waves.
  • the receiving antenna of the millimeter-wave radar sensor starts to receive the echo of electromagnetic waves. After the transmission is completed, the reception of the echo also ends, and the lidar sensor also rotates and scans the same field of view. In this way, the sampling signals of the two are synchronized in time.
  • the above-mentioned time synchronization is described by taking the lidar sensor triggering the millimeter-wave radar sensor as an example.
  • the time synchronization can also be triggered by the millimeter-wave radar sensor.
  • the millimeter-wave Each time the radar sensor sends a continuous electromagnetic wave, it sends a synchronous trigger signal to the lidar sensor. After the lidar sensor receives the synchronous trigger signal, it starts to perform the detection operation.
  • Spatial synchronization refers to the determination of the spatial transformation matrix through multiple detections of the same test target by multiple detection sensors.
  • the laser sampling signal and the millimeter-wave sampling signal can be processed spatially synchronously based on the spatial transformation matrix, and then, based on the spatially synchronously processed laser sampling signal and millimeter-wave sampling signals to perform subsequent operations.
  • the frequency bands of the various detection sensors are different, their reflectivity is also different, so the difference in signal amplitude of the sampling signals of multiple sensors may be relatively large.
  • the sensor sampling signals of each detection sensor may be subjected to regularization processing, so as to convert the signal amplitudes of each sensor sampling signal into a uniform amplitude range.
  • each sensor sampling signal can be regularized by the following formula (4):
  • V norm represents the normalized signal amplitude
  • V represents the original signal amplitude
  • V min represents the minimum signal amplitude in a frame of detection data
  • V max represents the maximum signal amplitude in a frame of detection data.
  • the electronic device executes subsequent steps based on time synchronization, space synchronization, and the multiple sensor sampling signals after regularization processing.
  • Step 102 Synchronize each sensor sampling signal in the multiple sensor sampling signals to the same feature dimension space to obtain multiple signal data, wherein the feature dimension space is set according to the detection dimension of each detection sensor and the system output requirements .
  • the detection dimension of the detection sensor is determined according to the characteristics of the detection sensor itself.
  • the detection dimension of the lidar sensor (here referred to as the first feature dimension space) includes the feature dimension of frame quantity feature, Scan beam characteristics, time characteristics, amplitude characteristics.
  • the frame quantity feature means that the laser sampling signal corresponds to multiple data frames
  • the scanning beam feature means that the laser sampling signal is obtained by scanning the scanning beam
  • the time feature means that the laser sampling signal corresponds to a sampling time
  • the amplitude feature means that the laser sampling signal has a corresponding sampling time.
  • the sampling signal has a certain amplitude.
  • the detection dimension of the millimeter-wave radar sensor includes feature dimensions including chirp features, radiation antenna features, time features, and amplitude features.
  • the chirp feature refers to the characteristic that the instantaneous pulse frequency changes with time
  • the radiation antenna feature means that the millimeter-wave sampling signal is obtained by emitting electromagnetic waves through the radiation antenna
  • the time feature means that the millimeter-wave sampling signal corresponds to the sampling time.
  • the amplitude characteristic means that the millimeter wave sampling signal has a certain amplitude.
  • System output requirements can be set according to actual needs.
  • the electronic device converts the sampling signal of the lidar sensor from the first feature dimension space to the feature dimension space, and converts the sampling signal of the millimeter-wave radar sensor from the second feature dimension space to the feature dimension space.
  • Step 103 Fusion processing is performed on multiple signal data to obtain a spatial signal description matrix, which can describe the signal echo quality of the target.
  • the electronic device may fuse multiple signal data by means of weighting, cross-correlation or binary hypothesis testing to obtain a spatial signal description matrix.
  • the spatial sampling densities of multiple detection sensors may be different.
  • the specific implementation of step 103 includes: when the sampling steps of multiple detection sensors are different At the same time, the sampling steps of multiple signal data are aligned by an interpolation method, so as to align the sampling steps of feature spaces of multiple detection sensors. Fusion processing is performed on the aligned multiple signal data to obtain a spatial signal description matrix.
  • Step 104 Output fusion data based on the spatial signal description matrix.
  • the peak value is extracted from the fused signal in the spatial signal description matrix, and the fused data is output according to the peak value extraction result.
  • the peak extraction result is output as fusion data.
  • the sampling signal of each detection sensor in multiple detection sensors in the same detection range is acquired at the same time to obtain multiple sensor sampling signals, and the working frequency bands of the multiple detection sensors are different. Synchronize each of the multiple sensor sampling signals to the same feature dimension space to obtain multiple signal data, wherein the feature dimension space is set according to the detection dimension of each detection sensor and the system output requirements. Multiple signal data are fused to obtain a spatial signal description matrix, which can describe the signal echo quality of the target. Based on the spatial signal description matrix, the fusion data is output. In this way, data fusion based on the original sensor sampling signal of the detection sensor can improve the robustness of detection while avoiding missed detection due to fusion based on point cloud data or target detection results, and improve the final fusion. Data Accuracy.
  • FIG. 2 is a schematic flowchart of a method for multi-sensor signal fusion according to another exemplary embodiment, and the method may be applied to the above-mentioned electronic device. As an example but not a limitation, the method may include the following implementation steps:
  • Step 201 Obtain the sampling signals of the lidar sensor and the millimeter-wave radar sensor for the same detection range at the same time, and obtain the laser sampling signal and the millimeter-wave sampling signal.
  • step 101 in the embodiment shown in FIG. 1 above.
  • Step 202 Synchronize the laser sampling signal and the millimeter wave sampling signal to the same feature dimension space respectively, to obtain the first laser signal data and the millimeter wave signal data.
  • the sampling signal of the lidar sensor is converted from the first feature dimension space to the feature dimension space
  • the feature dimensions included in the first feature dimension space are frame amount features, scanning beam features, and time features
  • the feature dimension space includes The feature dimensions are velocity features, angle features, and distance features.
  • Step 203 Determine at least one evaluation parameter capable of evaluating the signal echo quality of the target based on each signal data in the plurality of signal data, and obtain multiple sets of evaluation parameters, and each set of evaluation parameters in the multiple sets of evaluation parameters Include at least one evaluation parameter.
  • the plurality of signal data includes first laser signal data and millimeter wave signal data.
  • step 203 may include: based on the millimeter wave signal data, determining the millimeter wave peak position and the millimeter wave echo energy corresponding to the target to obtain a set of evaluation parameters. Based on the first laser signal data, determine the echo front position and laser peak saturation corresponding to the target, and obtain another set of evaluation parameters.
  • the echo front position refers to the position detected by the laser radar sensor when the target is first detected by the laser radar sensor. Position, laser peak saturation is used to indicate the confidence that the target is a true target.
  • the millimeter-wave peak position of the target is the position with the highest probability of occurrence of the target, that is, the target has the highest probability of appearing at the millimeter-wave peak position.
  • the mmWave peak position is an unbiased estimate of the target's position.
  • the first angular distance map may be determined based on the millimeter wave signal data, and then the millimeter wave peak position may be determined based on the first angular distance map.
  • the first angular distance map is a spectrum map of the millimeter wave sampled signal.
  • 3D-FFT Fast Fourier Transformation, Fast Fourier Transformation
  • processing may be performed on the millimeter wave sampled signal to obtain the first angle distance map.
  • the abscissa of the first angle-distance graph is the angle, and the ordinate is the distance, corresponding to the spatial sampling signal data in the polar coordinate system.
  • wavelet transform processing may also be performed on the millimeter wave sampled signal to determine the first angle-distance map.
  • subspace transformation processing may also be performed on the millimeter wave sampled signal, so as to determine the first angle distance map.
  • the millimeter wave peak of the target based on the first angle distance map, find the millimeter wave peak of the target through 2D-CFAR (Two Dimensional-Constant False-Alarm Rate, two-dimensional constant false alarm algorithm), and determine the angle and angle corresponding to the millimeter wave peak distance, and then determine the millimeter wave peak position based on the angle and distance.
  • the position of the millimeter wave peak can be determined by the following formula (5) according to the angle and distance:
  • x 1 represents the abscissa of the millimeter wave peak position
  • y 1 represents the vertical coordinate of the millimeter wave peak position
  • ⁇ represents the distance
  • ⁇ represents the angle
  • the electronic device can determine the millimeter wave echo energy of the target by integrating the echo value of the target based on the first angle distance map.
  • the electronic device regards the position of the millimeter wave peak and the energy of the millimeter wave echo as a set of evaluation parameters.
  • the electronic device determines the second angle distance map based on the first laser signal data, and then determines the echo front position and laser peak saturation corresponding to the target based on the second angle distance map.
  • the electronic device may directly construct the second angle distance map according to the first laser signal data.
  • the abscissa of the second angle distance graph is the angle
  • the ordinate is the distance.
  • FIG. 3 is a schematic diagram of an angle-distance diagram according to an exemplary embodiment, wherein the white highlighted area is detection data of a lidar sensor. At the position of the value 20 in the ordinate, a straight line from left to right is the detection data of the millimeter wave radar sensor.
  • the electronic device determines the corresponding position according to the target's echo, which is called the echo front position. For example, referring to FIG. 3 , assuming that the target is 31 in FIG. 3 , the position of the echo front of the target 31 is shown as 32 in FIG. 3 . Since the laser radar sensor uses the TOF method for distance measurement, the embodiment of the present application extracts the echo front position, and uses the echo front position as the unbiased estimation of the target position.
  • the waveform of each angle of the lidar sensor is extracted by 1D-CFAR (One Dimensional-Constant False-Alarm Rate, one-dimensional constant false alarm algorithm), and then determined The front position of the wave crest is used to obtain the front position of the echo.
  • 1D-CFAR One Dimensional-Constant False-Alarm Rate, one-dimensional constant false alarm algorithm
  • the laser peak saturation is used to indicate the characteristics of the laser echo, and can be used as an evaluation coefficient of the laser echo quality.
  • the specific implementation of determining the laser peak saturation of the target detected by the lidar sensor based on the second angle distance map may include: based on the second angle distance map, obtaining the laser echo value and laser echo value corresponding to the target Pulse Width.
  • the laser echo value is integrated to obtain the laser echo energy.
  • the value obtained by dividing the laser echo energy by the laser echo pulse width is used as the laser peak saturation corresponding to the target.
  • the laser echo pulse width corresponding to the target is determined through a dynamic threshold method.
  • the above-mentioned method of determining the laser peak saturation of the target detected by the lidar sensor based on the second angle distance map is only exemplary, and in another embodiment, its specific implementation may also include: based on the second angle Distance map, to obtain the laser echo value and laser echo pulse width corresponding to the target.
  • the laser echo value is integrated to obtain the laser echo energy. Determine the theoretical laser echo energy at the laser echo pulse width. The value obtained after dividing the laser echo energy by the theoretical laser echo energy is used as the laser peak saturation corresponding to the target.
  • Step 204 Perform signal fusion processing based on multiple sets of evaluation parameter sets to obtain a spatial signal description matrix.
  • step 204 may include: based on the millimeter wave peak position and the millimeter wave echo energy, determining the variance of the millimeter wave peak position to obtain the first variance, the first variance is used to indicate that the target is in The probability distribution at the position of the mmWave peak. Based on the position of the echo front and the saturation of the laser peak, the variance of the position of the echo front is determined to obtain the second variance, which is used to indicate the probability distribution of the target at the position of the echo front. Based on the position of the millimeter wave peak, the position of the echo front, the first variance, and the second variance, the spatial signal description matrix is determined.
  • the variance of the millimeter wave peak position is determined by the following formula (6):
  • ⁇ 1 is the first variance
  • ⁇ 1 is the millimeter wave peak position, including (x 1 , y 1 )
  • feature represents the millimeter wave echo energy
  • a and B are adjustable parameters, which can be set according to actual needs .
  • the variance of the echo front position is determined by the following formula (7):
  • ⁇ 2 is the second variance
  • ⁇ 2 is the position of the echo front, including (x 2 , y 2 )
  • power indicates the laser peak saturation
  • the spatial signal description matrix is determined through the following formulas (1) to (3):
  • ⁇ 12 is the joint position, including (x 12 , y 12 ), ⁇ 1 is the position of the millimeter wave crest, including (x 1 , y 1 ), ⁇ 2 is the position of the echo front, including (x 2 , y 2 ) , ⁇ 1 is the first variance, ⁇ 2 is the second variance, (x, y) is the variable, ⁇ 12 is the joint confidence, and V is the joint confidence distribution.
  • the electronic device can determine the joint confidence distribution map based on V, and then convert the joint confidence distribution map to obtain the spatial signal description matrix, that is, use the spatial signal description matrix to represent the joint confidence distribution map.
  • steps 203 to 204 are an exemplary implementation manner of performing fusion processing on multiple signal data.
  • Step 205 Output fusion data based on the spatial signal description matrix.
  • step 205 may include: determining, from the spatial signal description matrix, a joint position whose joint confidence is greater than a confidence threshold.
  • the joint position and the corresponding joint confidence of which the joint confidence is greater than the confidence threshold are output as fusion data.
  • the confidence threshold may be set by the user according to actual requirements, or may be set by default by the electronic device, which is not limited in this embodiment of the present application.
  • the joint position is the position with the highest probability of target occurrence detected by the millimeter-wave radar sensor and the lidar sensor, when the joint confidence of the joint position is greater than the confidence threshold, it means that the joint position is the position with the highest probability of target occurrence
  • the confidence is relatively high, that is, the reliability is relatively high, so the electronic device outputs the joint position and its joint confidence as fusion data.
  • the output fusion data may also include other detection data of the target.
  • the specific implementation of step 205 may also include determining, from the spatial signal description matrix, a joint position whose joint confidence is greater than a confidence threshold. Output the joint position, corresponding joint confidence, and speed whose joint confidence is greater than the confidence threshold as fusion data.
  • the speed refers to the speed detected by the millimeter wave radar sensor. That is, the output point cloud data may also include the speed detected by the millimeter wave radar sensor.
  • each joint position in the spatial signal description matrix and the joint confidence of each joint position can also be output as point cloud data, and when the point cloud data is used later, it can be based on the joint confidence of each joint position Combined confidence for screening.
  • the confidence map of the signal of the millimeter-wave radar sensor and the signal of the laser radar sensor can also be drawn confidence map.
  • the confidence distribution of the signal of the millimeter-wave radar sensor can be determined by the following formula (8) and formula (9):
  • V1 represents the confidence distribution of the signal of the millimeter-wave radar sensor.
  • the confidence map of the signal of the millimeter-wave radar sensor can be drawn based on the V 1 obtained above.
  • the confidence distribution of the signal of the lidar sensor can be determined by the following formula (10) and formula (11):
  • V 2 represents the confidence distribution of the signal of the lidar sensor
  • ⁇ 0 is the ranging distance of the lidar sensor
  • the confidence map of the signal of the lidar sensor can be drawn.
  • the millimeter wave sampling signal and the laser sampling signal for the same detection range are acquired at the same time. Synchronize the millimeter wave sampling signal and the laser sampling signal into the same feature dimension space.
  • the first angular distance map is obtained according to the synchronized millimeter wave sampling signal
  • the second angular distance map is obtained according to the synchronized laser sampling signal.
  • the first angle distance map and the second angle distance map are fused to obtain the spatial signal description matrix.
  • the spatial signal description matrix is used to describe the joint position of the target and the joint confidence of the joint position.
  • the joint position is the target with the highest probability of occurrence s position.
  • fusion processing based on the original sampling signal compared to fusion based on point cloud data or detection results, can enhance the sampling signal of fuzzy but real targets, and avoid filtering out such targets by mistake.
  • the fusion data is output. Since the spatial signal description matrix describes the joint position of the target and the joint confidence of the joint position, the accuracy of the finally generated fusion data can be improved.
  • FIG. 4 is a schematic flowchart of a method for multi-sensor signal fusion according to another exemplary embodiment, and the method may be applied to the above-mentioned electronic device. As an example but not a limitation, the method may include the following implementation steps:
  • steps 401 to 402 reference may be made to steps 201 to 202 above.
  • Step 403 Carry out position association based on multiple signal data.
  • the plurality of signal data includes first laser signal data and millimeter wave signal data.
  • the electronic device In order to determine which signal data among the first laser signal data and the millimeter wave signal data are of the same target, the electronic device performs position correlation processing based on the first laser signal data and the millimeter wave signal data.
  • a bipartite graph matching method may be used for location association.
  • the bipartite graph matching method may be the Hungarian algorithm, etc., which is not limited in this embodiment of the present application.
  • the electronic device can filter the static targets in the sensor signal data to preserve The signal data of the dynamic target is used for subsequent fusion processing on the signal data of the dynamic target.
  • the electronic device filters out the signal data of the static target detected by the lidar sensor from the first laser signal data to obtain the first dynamic target signal data, and filters out the signal data of the millimeter wave radar sensor from the millimeter wave signal data.
  • the signal data of the static target is used to obtain the signal data of the second dynamic target.
  • the electronic device performs filtering processing on the signal data of the static target in the millimeter wave signal data of the millimeter wave radar sensor, and performs filtering processing on the first laser signal data of the laser radar sensor The signal data of the static target is filtered out.
  • the millimeter wave signal data includes velocity.
  • the specific implementation of filtering out the signal data of the static target detected by the millimeter-wave radar sensor from the millimeter-wave signal data to obtain the second dynamic target signal data may include: determining the signal data whose speed is less than the speed threshold from the millimeter-wave signal data . Signal data with velocities less than a velocity threshold are removed from mmWave signal data. The millimeter wave signal data obtained after the deletion processing is determined as the second dynamic target signal data.
  • the speed threshold may be set by the user according to actual needs, or may be set by default by the electronic device, which is not limited in this embodiment of the present application.
  • the target may not move or move slightly, and in this case the target can be determined as a static target.
  • the electronic device deletes the signal data of the target from the millimeter wave signal data. In this way, the signal data of the static target in the millimeter wave signal data can be filtered out, and the rest is the signal data of the dynamic target detected by the millimeter wave radar sensor, that is, the second dynamic target signal data is obtained .
  • a target whose speed is less than a speed threshold may also be determined as a static target. Examples are not limited to this.
  • the specific realization of obtaining the first dynamic target signal data by filtering out the signal data of the static target detected by the laser radar sensor from the first laser signal data may include: acquiring the second laser signal data, the second laser signal The data is one frame of laser signal data adjacent to the first laser signal data. The signal data difference obtained after subtracting the first laser signal data from the second laser signal data is used as the first dynamic target signal data.
  • the second laser signal data may be the last frame of laser signal data of the first laser signal data.
  • the second laser signal data may also be the next frame of laser signal data of the first laser signal data.
  • the signal data of the static target can be filtered out after subtracting the signal data of two adjacent frames of the lidar sensor. Therefore, after the echo difference of adjacent frames is performed on the first laser signal data, the signal data of the dynamic target detected by the laser radar sensor can be determined, that is, the first dynamic target signal data can be obtained.
  • laser signal data that is separated from the first laser signal data by a preset number of frames may also be acquired to obtain third laser signal data. Afterwards, the third laser signal data is subtracted from the first laser signal data to determine the first dynamic target signal data.
  • the preset quantity can be set according to actual needs.
  • the laser signal data of the previous frame or the next frame of the second laser signal data can be obtained to obtain the third laser signal data, for example, if the second laser signal data is the laser signal data of the previous frame of the first laser signal data signal data, then the previous frame of the second laser signal data can be used as the third laser signal data, and for another example, if the second laser signal data is the next frame signal data of the first laser signal data, the second laser signal data can be used The next frame of signal data is used as the third laser signal data.
  • This embodiment of the present application does not limit it.
  • the specific implementation of performing location association based on multiple signal data includes: performing location association based on the first dynamic target signal data and the second dynamic target signal data.
  • the association method can adopt the bipartite graph matching method.
  • Step 404 According to the location correlation result, determine the signal data of the target from each of the multiple signal data respectively, and obtain multiple target signal data of the target.
  • the target detected by the lidar sensor and the millimeter-wave radar sensor can be determined at the same time, or the same target detected by the lidar sensor and the millimeter-wave radar sensor can be determined.
  • the signal data of the associated target is respectively acquired from each sensor signal data to obtain a plurality of target signal data of the target. .
  • the number of associated targets may be one, or may be multiple.
  • the signal data of each associated target may be fused according to the following method.
  • multiple detection sensors include a millimeter-wave radar sensor and a visible light detection sensor
  • the same target detected by the two can also be determined by sampling position matching, and then, from the millimeter-wave signal data of the millimeter-wave radar sensor The signal data of the same target is determined, and the signal data of the same target is determined from the visible light image of the visible light detection sensor to obtain multiple signal data of the target.
  • the plurality of detection sensors include a millimeter wave radar sensor and an infrared detection sensor, which will not be repeated here.
  • Step 405 Perform fusion processing on multiple target signal data to obtain a spatial signal description matrix.
  • a plurality of target signal data are arranged in a certain arrangement manner to obtain a spatial signal description matrix. For example, arrange the target signal data belonging to the first dynamic target signal data among the multiple target signal data in the first column, and arrange the target signal data belonging to the second dynamic target signal data among the multiple target signal data in the second column, Get the spatial signal description matrix.
  • Step 406 Output fusion data based on the spatial signal description matrix.
  • step 406 may include: acquiring an angle of the target in the first dynamic target signal data, and acquiring a distance of the target in the second dynamic target signal data. Determine angles and distances as fused data.
  • the measurement of the target angle of arrival by the millimeter-wave radar sensor can be provided by the lidar sensor.
  • An angle in the dynamic target signal data is output.
  • the ranging information of the millimeter-wave radar sensor is relatively accurate, and the ranging information of the laser radar sensor will be affected by noise, so the ranging information of the target is provided by the millimeter-wave radar sensor, so the second dynamic target signal of the millimeter-wave radar is transmitted to the sensor.
  • the distance in the data is taken as output, so that the fused data can be obtained.
  • the distance and angle of the static target in the first laser signal data are output. That is, for static targets, the detection results of the lidar sensor can be directly output.
  • FIG. 3 is a schematic diagram of signal data according to an exemplary embodiment, wherein the abscissa is an angle, from -10 degrees to 10 degrees, and the ordinate is a distance, from 0 meters to 80 meters.
  • the white brightness in Figure 3 (here white is taken as an example, and other colors, such as yellow) area is the signal data of the laser radar sensor.
  • the distance is about 20 meters, and a line from left to right is the signal data of the millimeter wave radar sensor. signal data. That is to say, the millimeter-wave radar sensor has detected the target at 20m, but it does not know which angle it is from, but the lidar sensor can clearly detect the outline of the target from around 5 degrees.
  • the distance of the target located at about 5 degrees and 20 meters can be given by the ranging result of the millimeter-wave radar sensor, and the laser radar sensor provides it from which angle to which angle at about 5 degrees.
  • the target that is not detected by the millimeter-wave radar sensor is a static target that is filtered out, and the detection results of the lidar sensor can provide distance and angle. In this way, the point cloud image shown in Figure 6 can be obtained.
  • step 406 may further include: acquiring the angle of the target in the first dynamic target signal data, and acquiring the distance and speed of the target in the second dynamic target signal data. The angle, distance, and speed of the target are determined as fusion data.
  • the sampling signal of each detection sensor in the multiple detection sensors in the same detection range is obtained at the same time to obtain multiple sensor sampling signals, and the working frequency bands of the multiple detection sensors are different. Synchronize multiple sensor sampling signals to the same feature dimension space respectively. Get multiple signal data. Based on the multiple signal data of the target, the multiple signal data are fused to obtain fused data. In this way, data fusion based on the original sensor sampling signal of the detection sensor can improve the robustness of detection while avoiding missed detection due to fusion based on point cloud data or target detection results, and improve the final fusion. Data Accuracy.
  • the signal data detected by the millimeter-wave radar sensor can be used as the visible light detection sensor when fusion is performed based on multiple signal data of the target. Additional channels in the signal data to obtain fused data.
  • the distance in the signal data of the millimeter-wave radar sensor can be used as an additional channel in the visible light image; for another example, the distance and angle in the signal data of the millimeter-wave radar sensor can be used as two additional channels in the visible light image aisle.
  • the signal data detected by the millimeter-wave radar sensor can be used as the signal detected by the infrared detection sensor Additional channels in the data, resulting in fused data.
  • the distance in the signal data of the millimeter-wave radar sensor can be used as an additional channel in the infrared image.
  • the distance and angle in the signal data of the millimeter-wave radar sensor can be used as two additional channels in the infrared image. aisle.
  • FIG. 7 is a schematic structural diagram of a multi-sensor signal fusion device according to an exemplary embodiment.
  • the device can be composed of software, hardware or both. A combination of the above can be referred to as part or all of the above-mentioned electronic equipment.
  • the device can include:
  • the acquisition module 710 is configured to acquire the sampling signal of each detection sensor in the multiple detection sensors in the same detection range at the same time, and obtain multiple sensor sampling signals, and the working frequency bands of the multiple detection sensors are different;
  • Synchronization module 720 configured to synchronize each sensor sampling signal in the plurality of sensor sampling signals to the same feature dimension space to obtain a plurality of signal data, wherein the feature dimension space is based on each detection sensor The detection dimension and system output requirements are set;
  • the fusion module 730 is used to perform fusion processing on the plurality of signal data to obtain a spatial signal description matrix, and the spatial signal description matrix can describe the signal echo quality of the target;
  • An output module 740 configured to output fusion data based on the spatial signal description matrix.
  • the fusion module 730 is used for:
  • the evaluation parameter set includes said at least one evaluation parameter
  • Signal fusion processing is performed based on the multiple sets of evaluation parameter sets.
  • the plurality of signal data includes first laser signal data and millimeter wave signal data; the fusion module 730 is used for:
  • the echo front position refers to the initial detection of the target by the laser radar sensor
  • the saturation of the laser peak is used to indicate the confidence that the target is a real target.
  • the fusion module 730 is used for:
  • the millimeter wave peak position Based on the millimeter wave peak position and the millimeter wave echo energy, determine the variance of the millimeter wave peak position to obtain a first variance, and the first variance is used to indicate that the target is at the millimeter wave peak The probability distribution at the location;
  • the variance of the echo front position Based on the echo front position and the laser peak saturation, determine the variance of the echo front position to obtain a second variance, the second variance is used to indicate the position of the target at the echo front position probability distribution;
  • the spatial signal description matrix is determined based on the millimeter wave peak position, the echo front position, the first variance, and the second variance.
  • the fusion module 730 is used for:
  • the spatial signal description matrix is determined through the following formulas (1) to (3):
  • the ⁇ 12 includes (x 12 , y 12 ), the ⁇ 1 is the position of the millimeter wave crest, including (x 1 , y 1 ), and the ⁇ 2 is the position of the echo front, including ( x 2 , y 2 ), the ⁇ 1 is the first variance, the ⁇ 2 is the second variance, the (x, y) is a variable, and the ⁇ 12 is the joint confidence, so
  • the above V is the joint confidence distribution.
  • the output module 740 is used to:
  • the fusion module 730 is used for:
  • the position correlation result determine the signal data of the target from each signal data in the plurality of signal data respectively, and obtain a plurality of target signal data of the target;
  • Fusion processing is performed based on the plurality of target signal data.
  • the plurality of detection sensors include lidar sensors and millimeter wave radar sensors, and the plurality of signal data includes first laser signal data and millimeter wave signal data;
  • the fusion module 730 is configured to: filter out the signal data of the static target detected by the lidar sensor from the first laser signal data to obtain the first dynamic target signal data, and filter out the signal data from the millimeter wave signal data. removing the signal data of the static target detected by the millimeter-wave radar sensor to obtain the second dynamic target signal data;
  • the specific implementation of performing position association by the fusion module 730 based on the plurality of signal data includes: performing position association based on the first dynamic target signal data and the second dynamic target signal data.
  • the millimeter wave signal data includes speed
  • the fusion module 730 is used to:
  • the millimeter wave signal data obtained after the deletion processing is determined as the second dynamic target signal data.
  • the fusion module 730 is used for:
  • the second laser signal data is a frame of laser signal data adjacent to the first laser signal data
  • a signal data difference obtained by subtracting the first laser signal data from the second laser signal data is used as the second dynamic target signal data.
  • the feature dimension space includes angle features and distance features
  • the output module 740 is used to:
  • the angle and the distance are determined as the fused data.
  • the output module 740 is also used for:
  • the synchronization module 720 is also used for:
  • the regularized sensor sampling signals are respectively synchronized to the feature dimension space.
  • the synchronization module 720 is also used for:
  • time synchronization refers to triggering the work of other detection sensors in the plurality of sensors by one detection sensor in the plurality of sensors
  • space synchronization refers to A spatial transformation matrix is determined by multiple detections of the same test target by the plurality of detection sensors.
  • the fusion module 730 is used for:
  • Fusion processing is performed on the plurality of aligned signal data.
  • the multiple detection sensors include lidar sensors and millimeter wave radar sensors
  • the synchronization module 720 is used for:
  • the feature dimensions included in the first feature dimension space are frame amount features, scanning beam features, and time features, and the feature dimensions included in the space are velocity features, angle features, and distance features;
  • the sampling signal of the millimeter-wave radar sensor is converted from the second feature dimension space to the feature dimension space, and the feature dimensions included in the second feature dimension space are chirp feature, radiation antenna feature, and time feature.
  • the sampling signal of each detection sensor in the multiple detection sensors in the same detection range is obtained at the same time to obtain multiple sensor sampling signals, and the working frequency bands of the multiple detection sensors are different. Synchronize each of the multiple sensor sampling signals to the same feature dimension space to obtain multiple signal data, wherein the feature dimension space is set according to the detection dimension of each detection sensor and the system output requirements. Multiple signal data are fused to obtain a spatial signal description matrix, which can describe the signal echo quality of the target. Based on the spatial signal description matrix, the fusion data is output. In this way, data fusion based on the original sensor sampling signal of the detection sensor can improve the robustness of detection while avoiding missed detection due to fusion based on point cloud data or target detection results, and improve the final fusion. the accuracy of the data.
  • FIG. 8 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.
  • the electronic device 8 of this embodiment includes: at least one processor 80 (only one is shown in FIG. 8 ), a memory 81, and stored in the memory 81 and can be used in the at least one processor 80.
  • the electronic device 8 may be computing devices such as desktop computers, notebooks, palmtop computers, and cloud servers.
  • the electronic device may include, but not limited to, a processor 80 and a memory 81 .
  • FIG. 8 is only an example of the electronic device 8, and does not constitute a limitation to the electronic device 8. It may include more or less components than shown in the figure, or combine certain components, or different components. , for example, may also include input and output devices, network access devices, and so on.
  • the so-called processor 80 can be a CPU (Central Processing Unit, central processing unit), and the processor 80 can also be other general processors, DSP (Digital Signal Processor, digital signal processor), ASIC (Application Specific Integrated Circuit, dedicated integrated circuit), FPGA (Field-Programmable Gate Array, off-the-shelf programmable gate array) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.
  • a general-purpose processor may be a microprocessor, or the processor may be any conventional processor, or the like.
  • the storage 81 may be an internal storage unit of the electronic device 8 in some embodiments, such as a hard disk or memory of the electronic device 8 .
  • the memory 81 may also be an external storage device of the electronic device 8 in other embodiments, such as a plug-in hard disk equipped on the electronic device 8, an SMC (Smart Media Card, a smart memory card), SD ( Secure Digital, Secure Digital) card, flash memory card (Flash Card), etc. Further, the memory 81 may also include both an internal storage unit of the electronic device 8 and an external storage device.
  • the memory 81 is used to store operating system, application program, boot loader (BootLoader), data and other programs, such as the program code of the computer program.
  • the memory 81 can also be used to temporarily store data that has been output or will be output.

Landscapes

  • Engineering & Computer Science (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Physics & Mathematics (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • General Physics & Mathematics (AREA)
  • Electromagnetism (AREA)
  • Radar Systems Or Details Thereof (AREA)

Abstract

本申请公开了一种多传感器信号融合的方法、装置、电子设备及存储介质，属于智能监控技术领域。包括：获取同一时刻的、针对同一探测范围的多个探测传感器中每个探测传感器的采样信号，多个探测传感器的工作频段不同。分别将多个传感器采样信号中的每个传感器采样信号同步至同一特征维度空间，得到多个信号数据，其中，特征维度空间是根据每个探测传感器的探测维度和系统输出要求设置的。对多个信号数据进行融合处理，得到空间信号描述矩阵，空间信号描述矩阵能够描述目标的信号回波质量。基于空间信号描述矩阵，输出融合数据。如此，基于探测传感器的原始的采样信号进行数据融合，避免融合出现漏检等情况，提高了最终生成的融合数据的准确度。

Description

多传感器信号融合的方法、装置、电子设备及存储介质
本申请要求于2021年10月26日在中国专利局提交的、申请号为202111251503.2、发明名称为“多传感器信号融合的方法、装置、电子设备及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及智能监控技术领域，特别涉及一种多传感器信号融合的方法、装置、电子设备及存储介质。
背景技术
探测传感器在智能监控技术领域得到广泛应用。常见的探测传感器包括但不限于激光雷达传感器、毫米波雷达传感器、可见光传感器。不同探测传感器的工作频段不同，使得不同探测传感器的探测性能有所差异，譬如，毫米波雷达传感器的穿透能力强于激光雷达传感器，但激光雷达传感器的探测精度高于毫米波雷达传感器。
由于单个探测传感器的探测性能存在一定的缺陷，所以通常采用多个传感器融合来提升探测的鲁棒性。目前的融合大多数是基于点云数据进行融合或者是基于目标检测结果进行融合，其中，这些点云数据或目标检测结果是经过一系列筛选等处理之后得到的。
然而，若单个探测传感器在探测时遇到挑战，譬如激光雷达传感器遇到雾霾、沙尘暴等恶劣天气，则会导致在筛选等处理过程中将一些模糊但真实的目标的相关数据筛选掉。如此，在融合时会出现漏检等情况，降低了最终生成的融合数据的准确度。
申请内容
本申请实施例提供了一种多传感器信号融合的方法、装置、电子设备及存储介质，可以解决相关技术中在融合时会出现漏检等情况，导致降低了最终生成的融合数据的准确度的问题。所述技术方案如下：
第一方面，提供了一种多传感器信号融合的方法，所述方法包括：
获取同一时刻的、针对同一探测范围的多个探测传感器中每个探测传感器的采样信号，得到多个传感器采样信号，所述多个探测传感器的工作频段不相同；
分别将所述多个传感器采样信号中的每个传感器采样信号同步至同一特征维度空间，得到多个信号数据，其中，所述特征维度空间是根据所述每个探测传感器的探测维度和系统输出要求设置的；
对所述多个信号数据进行融合处理，得到空间信号描述矩阵，所述空间信号描述矩阵能够描述目标的信号回波质量；
基于所述空间信号描述矩阵，输出融合数据。
作为本申请的一个示例，所述对所述多个信号数据进行融合处理，包括：
分别基于所述多个信号数据中的每个信号数据，确定能够评价所述目标的信号回波质量的至少一个评价参数，得到多组评价参数集合，所述多组评价参数集合中的每组评价参数集合包括所述至少一个评价参数；
基于所述多组评价参数集合进行信号融合处理。
作为本申请的一个示例，所述多个信号数据包括第一激光信号数据和毫米波信号数据；
所述分别基于所述多个信号数据中的每个信号数据，确定能够评价所述目标的信号回波质量的至少一个评价参数，得到多组评价参数集合，包括：
基于所述毫米波信号数据，确定所述目标对应的毫米波波峰位置和毫米波回波能量，得到一组评价参数集合；
基于所述第一激光信号数据，确定所述目标对应的回波前沿位置和激光波峰饱和度，得到另一组评价参数集合，所述回波前沿位置是指所述目标被激光雷达传感器初次探测到时所述激光雷达传感器探测出的位置，所述激光波峰饱和度用于指示所述目标为真实目标的置信度。
作为本申请的一个示例，所述基于所述多组评价参数集合进行信号融合处理，包括：
基于所述毫米波波峰位置和所述毫米波回波能量，确定所述毫米波波峰位置的方差，得到第一方差，所述第一方差用于指示所述目标在所述毫米波波峰位置处的概率分布情况；
基于所述回波前沿位置和所述激光波峰饱和度，确定所述回波前沿位置的方差，得到第二方差，所述第二方差用于指示所述目标在所述回波前沿位置处的概率分布情况；
基于所述毫米波波峰位置、所述回波前沿位置、所述第一方差、所述第二方差，确定所述空间信号描述矩阵。
作为本申请的一个示例，所述基于所述毫米波波峰位置、所述回波前沿位置、所述第一方差、所述第二方差，确定所述空间信号描述矩阵，包括：
基于所述毫米波波峰位置、所述回波前沿位置、所述第一方差、所述第二方差，通过如下公式(1)至(3)，确定所述空间信号描述矩阵：
Figure PCTCN2022127075-appb-000001
Figure PCTCN2022127075-appb-000002
Figure PCTCN2022127075-appb-000003
其中，所述μ 12包括(x 12,y 12)，所述μ 1为所述毫米波波峰位置，包括(x 1,y 1)，所述μ 2为所述回波前沿位置，包括(x 2,y 2)，所述σ 1为所述第一方差，所述σ 2为所述第二方差，所述(x,y)为变量，所述σ 12为联合置信度，所述V为联合置信度分布。
作为本申请的一个示例，所述基于所述空间信号描述矩阵，输出融合数据，包括：
从所述空间信号描述矩阵中，确定联合置信度大于置信度阈值的联合位置；
将所述联合置信度大于置信度阈值的联合位置和对应的联合置信度作为所述融合数据输出。
作为本申请的一个示例，所述对所述多个信号数据进行融合处理，包括：
基于所述多个信号数据进行位置关联；
根据位置关联结果，分别从所述多个信号数据中的每个信号数据中，确定所述目标的信号数据，得到所述目标的多个目标信号数据；
基于所述多个目标信号数据进行融合处理。
作为本申请的一个示例，所述多个探测传感器包括激光雷达传感器和毫米波雷达传感器，所述多个信号数据包括第一激光信号数据和毫米波信号数据；
所述基于所述多个信号数据进行位置关联之前，还包括：
从所述第一激光信号数据中滤除所述激光雷达传感器探测的静态目标的信号数据，得到第一动态目标信号数据，以及从所述毫米波信号数据中滤除所述毫米波雷达传感器探测的静态目标的信号数据，得到第二动态目标信号数据；
所述基于所述多个信号数据进行位置关联，包括：
基于所述第一动态目标信号数据和所述第二动态目标信号数据进行位置关联。
作为本申请的一个示例，所述毫米波信号数据中包括速度，所述从所述毫米波信号数据中滤除所述毫米波雷达传感器探测的静态目标的信号数据，得到第二动态目标信号数据，包括：
从所述毫米波信号数据中确定速度小于速度阈值的信号数据；
从所述毫米波信号数据中删除所述速度小于所述速度阈值的信号数据；
将经过删除处理后得到的毫米波信号数据确定为所述第二动态目标信号数据。
作为本申请的一个示例，所述从所述第一激光信号数据中滤除所述激光雷达传感器探测的静态目标 的信号数据，得到第一动态目标信号数据，包括：
获取第二激光信号数据，所述第二激光信号数据是与所述第一激光信号数据相邻的一帧激光信号数据；
将所述第一激光信号数据与所述第二激光信号数据相减后得到的信号数据差值作为所述第二动态目标信号数据。
作为本申请的一个示例，所述特征维度空间包括角度特征和距离特征，所述基于所述空间信号描述矩阵，输出融合数据，包括：
获取所述第一动态目标信号数据中所述目标的角度、以及获取所述第二动态目标信号数据中所述目标的距离；
将所述角度和所述距离确定为所述融合数据。
作为本申请的一个示例，所述方法还包括：
输出所述第一激光信号数据中所述静态目标的距离和角度。
作为本申请的一个示例，所述分别将所述多个传感器采样信号中的每个传感器采样信号同步至同一特征维度空间之前，还包括：
分别对所述多个传感器采样信号中的每个传感器采样信号进行正则化处理；
所述分别将所述多个传感器采样信号中的每个传感器采样信号同步至同一特征维度空间，包括：分别将经正则化处理后的所述每个传感器采样信号同步至所述特征维度空间。
作为本申请的一个示例，所述方法还包括：
对所述多个探测传感器进行时间同步和空间同步，其中，所述时间同步是指通过所述多个传感器中的一个探测传感器触发所述多个传感器中的其他探测传感器工作，空间同步是指通过所述多个探测传感器对相同测试目标的多次探测确定空间转换矩阵。
作为本申请的一个示例，所述对所述多个信号数据进行融合处理，包括：
当所述多个探测传感器的采样步长不相同时，通过插值法对齐所述多个信号数据的采样步长；
对经对齐处理后的所述多个信号数据进行融合处理。
作为本申请的一个示例，所述多个探测传感器包括激光雷达传感器和毫米波雷达传感器；
所述分别将所述多个传感器采样信号中的每个传感器采样信号同步至同一特征维度空间，包括：
将所述激光雷达传感器的采样信号从第一特征维度空间转换至所述特征维度空间，所述第一特征维度空间包括的特征维度为帧量特征、扫描光束特征、时间特征，所述特征维度空间包括的特征维度为速度特征、角度特征、距离特征；
将所述毫米波雷达传感器的采样信号从第二特征维度空间转换至所述特征维度空间，所述第二特征维度空间包括的特征维度为啁啾特征、辐射天线特征、时间特征。
第二方面，提供了一种多传感器信号融合的装置，所述装置包括：
获取模块，用于获取同一时刻的、针对同一探测范围的多个探测传感器中每个探测传感器的采样信号，得到多个传感器采样信号，所述多个探测传感器的工作频段不相同；
同步模块，用于分别将所述多个传感器采样信号中的每个传感器采样信号同步至同一特征维度空间，得到多个信号数据，其中，所述特征维度空间是根据所述每个探测传感器的探测维度和系统输出要求设置的；
融合模块，用于对所述多个信号数据进行融合处理，得到空间信号描述矩阵，所述空间信号描述矩阵能够描述目标的信号回波质量；
输出模块，用于基于所述空间信号描述矩阵，输出融合数据。
作为本申请的一个示例，所述融合模块用于：
分别基于所述多个信号数据中的每个信号数据，确定能够评价所述目标的信号回波质量的至少一个评价参数，得到多组评价参数集合，所述多组评价参数集合中的每组评价参数集合包括所述至少一个评价参数；
基于所述多组评价参数集合进行信号融合处理。
作为本申请的一个示例，所述多个信号数据包括第一激光信号数据和毫米波信号数据；所述融合模 块用于：
所述分别基于所述多个信号数据中的每个信号数据，确定能够评价所述目标的信号回波质量的至少一个评价参数，得到多组评价参数集合，包括：
基于所述毫米波信号数据，确定所述目标对应的毫米波波峰位置和毫米波回波能量，得到一组评价参数集合；
基于所述第一激光信号数据，确定所述目标对应的回波前沿位置和激光波峰饱和度，得到另一组评价参数集合，所述回波前沿位置是指所述目标被激光雷达传感器初次探测到时所述激光雷达传感器探测出的位置，所述激光波峰饱和度用于指示所述目标为真实目标的置信度。
作为本申请的一个示例，所述融合模块用于：
基于所述毫米波波峰位置和所述毫米波回波能量，确定所述毫米波波峰位置的方差，得到第一方差，所述第一方差用于指示所述目标在所述毫米波波峰位置处的概率分布情况；
基于所述回波前沿位置和所述激光波峰饱和度，确定所述回波前沿位置的方差，得到第二方差，所述第二方差用于指示所述目标在所述回波前沿位置处的概率分布情况；
基于所述毫米波波峰位置、所述回波前沿位置、所述第一方差、所述第二方差，确定所述空间信号描述矩阵。
作为本申请的一个示例，所述融合模块用于：
基于所述毫米波波峰位置、所述回波前沿位置、所述第一方差、所述第二方差，通过如下公式(1)至(3)，确定所述空间信号描述矩阵：
Figure PCTCN2022127075-appb-000004
Figure PCTCN2022127075-appb-000005
Figure PCTCN2022127075-appb-000006
其中，所述μ 12包括(x 12,y 12)，所述μ 1为所述毫米波波峰位置，包括(x 1,y 1)，所述μ 2为所述回波前沿位置，包括(x 2,y 2)，所述σ 1为所述第一方差，所述σ 2为所述第二方差，所述(x,y)为变量，所述σ 12为联合置信度，所述V为联合置信度分布。
作为本申请的一个示例，所述输出模块用于：
从所述空间信号描述矩阵中，确定联合置信度大于置信度阈值的联合位置；
将所述联合置信度大于置信度阈值的联合位置和对应的联合置信度作为所述融合数据输出。
作为本申请的一个示例，所述融合模块用于：
基于所述多个信号数据进行位置关联；
根据位置关联结果，分别从所述多个信号数据中的每个信号数据中，确定所述目标的信号数据，得到所述目标的多个目标信号数据；
基于所述多个目标信号数据进行融合处理。
作为本申请的一个示例，所述多个探测传感器包括激光雷达传感器和毫米波雷达传感器，所述多个信号数据包括第一激光信号数据和毫米波信号数据；
所述融合模块用于：从所述第一激光信号数据中滤除所述激光雷达传感器探测的静态目标的信号数据，得到第一动态目标信号数据，以及从所述毫米波信号数据中滤除所述毫米波雷达传感器探测的静态目标的信号数据，得到第二动态目标信号数据；
所述融合模块基于所述多个信号数据进行位置关联的具体实现包括：基于所述第一动态目标信号数 据和所述第二动态目标信号数据进行位置关联。
作为本申请的一个示例，所述毫米波信号数据中包括速度，所述融合模块用于：
从所述毫米波信号数据中删除所述速度小于所述速度阈值的信号数据；
将经过删除处理后得到的毫米波信号数据确定为所述第二动态目标信号数据。
作为本申请的一个示例，所述融合模块用于：
获取第二激光信号数据，所述第二激光信号数据是与所述第一激光信号数据相邻的一帧激光信号数据；
将所述第一激光信号数据与所述第二激光信号数据相减后得到的信号数据差值作为所述第二动态目标信号数据。
作为本申请的一个示例，所述特征维度空间包括角度特征和距离特征，所述输出模块用于：
获取所述第一动态目标信号数据中所述目标的角度、以及获取所述第二动态目标信号数据中所述目标的距离；
将所述角度和所述距离确定为所述融合数据。
作为本申请的一个示例，所述输出模块还用于：
输出所述第一激光信号数据中所述静态目标的距离和角度。
作为本申请的一个示例，所述同步模块还用于：
分别对所述多个传感器采样信号中的每个传感器采样信号进行正则化处理；
分别将经正则化处理后的所述每个传感器采样信号同步至所述特征维度空间。
作为本申请的一个示例，所述同步模块还用于：
对所述多个探测传感器进行时间同步和空间同步，其中，所述时间同步是指通过所述多个传感器中的一个探测传感器触发所述多个传感器中的其他探测传感器工作，空间同步是指通过所述多个探测传感器对相同测试目标的多次探测确定空间转换矩阵。
作为本申请的一个示例，所述融合模块用于：
当所述多个探测传感器的采样步长不相同时，通过插值法对齐所述多个信号数据的采样步长；
对经对齐处理后的所述多个信号数据进行融合处理。
作为本申请的一个示例，所述多个探测传感器包括激光雷达传感器和毫米波雷达传感器；
所述同步模块用于：
将所述激光雷达传感器的采样信号从第一特征维度空间转换至所述特征维度空间，所述第一特征维度空间包括的特征维度为帧量特征、扫描光束特征、时间特征，所述特征维度空间包括的特征维度为速度特征、角度特征、距离特征；
将所述毫米波雷达传感器的采样信号从第二特征维度空间转换至所述特征维度空间，所述第二特征维度空间包括的特征维度为啁啾特征、辐射天线特征、时间特征。
第三方面，提供了一种电子设备，包括存储器、处理器以及存储在所述存储器中并可在所述处理器上运行的计算机程序，所述处理器执行所述计算机程序时实现上述第一方面任一项所述的方法。
第四方面，提供了一种计算机可读存储介质，所述计算机可读存储介质上存储有指令，所述指令被处理器执行时实现上述第一方面任一项所述的方法。
第五方面，提供了一种包含指令的计算机程序产品，当其在计算机上运行时，使得计算机执行上述第一方面任一项所述的方法。
本申请实施例提供的技术方案带来的有益效果是：
获取同一时刻的、针对同一探测范围的多个探测传感器中每个探测传感器的采样信号，得到多个传感器采样信号，多个探测传感器的工作频段不相同。分别将多个传感器采样信号中的每个传感器采样信号同步至同一特征维度空间，得到多个信号数据，其中特征维度空间是根据每个探测传感器的探测维度和系统输出要求设置的。对多个信号数据进行融合处理，得到空间信号描述矩阵，空间信号描述矩阵能够描述目标的信号回波质量。基于空间信号描述矩阵，输出融合数据。如此，基于探测传感器的原始的传感器采样信号进行数据融合，在提高探测的鲁棒性的同时，避免因基于点云数据或者基于目标检测结果融合导致出现漏检等情况，提高了最终生成的融合数据的准确度。
附图说明
为了更清楚地说明本申请实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1是根据一示例性实施例示出的一种多传感器信号融合的方法流程图；
图2是根据另一示例性实施例示出的一种多传感器信号融合的方法流程图；
图3是根据一示例性实施例示出的一种采样信号的示意图；
图4是根据另一示例性实施例示出的一种多传感器信号融合的方法流程图；
图5是根据另一示例性实施例示出的一种多传感器信号融合的方法流程图；
图6是根据一示例性实施例示出的一种点云图的示意图；
图7是根据一示例性实施例示出的一种多传感器信号融合的装置的结构示意图；
图8是根据一示例性实施例示出的一种电子设备的结构示意图。
具体实施方式
为使本申请的目的、技术方案和优点更加清楚，下面将结合附图对本申请实施方式作进一步地详细描述。
应当理解的是，本申请提及的“多个”是指两个或两个以上。在本申请的描述中，除非另有说明，“/”表示或的意思，例如，A/B可以表示A或B；本文中的“和/或”仅仅是一种描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。另外，为了便于清楚描述本申请的技术方案，采用了“第一”、“第二”等字样对功能和作用基本相同的相同项或相似项进行区分。本领域技术人员可以理解“第一”、“第二”等字样并不对数量和执行次序进行限定，并且“第一”、“第二”等字样也并不限定一定不同。
在对本申请实施例提供的多传感器信号融合的方法进行详细介绍之前，先对本申请实施例涉及的术语进行简单介绍。
激光雷达传感器：通过激光扫描的方式，在探测范围内能够还原出周围环境的三维感知信息。其工作原理是，采用TOF(Time of flight，飞行时间测距)法进行测距，并在接收光电二极管后端进行AD(Analog-to-digital，模拟到数据)等时采样，在完成每个角度的采样之后，将采样信号幅值序列从网口输出，得到激光采样信号。激光雷达传感器可以实现精细的三维感知，能够达到0.2度的角分辨率。然而，当周围环境中出现肉眼可见的噪声时，激光雷达传感器的探测能力将会急剧下降。例如，在出现团雾、暴雨、大雪、沙尘等天气时，由于空气中的粒子聚合物的直径与激光雷达传感器的波长相当，并且粒子的量足够多，使得激光雷达传感器扫描的激光在入射到这些粒子聚合体中时发生衰减和后向散射。一方面产生了对于空气中粒子聚合体的误检测回波，另一方面降低了激光的传输功率，从而会缩小激光雷达传感器的探测范围。
毫米波雷达传感器：通过发射电磁波对周围环境进行感知。其工作原理是，采用包括FMCW(Frequency Modulated Continuous Wave，调频连续波)在内的调制方法进行空间测量，从射频天线上发送和接收基频为24GHz或77GHz的连续调制电磁波。对接收天线接收到的回波进行AD采样，存储到寄存器，在完成一帧信号接收后从网络接口输出，得到毫米波采样信号。毫米波雷达传感器具有受噪声影响较小的特点，也即具有较强的抗干扰能力。然而毫米波雷达传感器的探测精度较低，能够达到1度的角分辨率。
时间同步：包括硬件时间同步和软件时间同步。在一个实施例中，硬件时间同步是指通过一者每隔固定时间向另一者发送同步触发信号，另一者接收到同步触发信号后执行探测操作。二者同时输出采样信号。软件时间同步是建立在两个探测传感器分别有固定的输出帧率的基础上，将二者输出的每一帧在同一时间坐标轴下进行帧号对齐和平移对齐。
空间同步：设置多个探测传感器中每个探测传感器的波段均可见的标志物(如金属标志物)，使用多个探测传感器同时对标志物所在空间进行采样，将标志物在空间上进行旋转平移对齐。重复进行采样和旋转平移的操作，即可得到多组采样数据。基于每组采样数据，分别解算出标志物的位置信息，得到多组位置信息。基于多组位置信息通过解方程组确定旋转矩阵和平移向量，得到空间转换矩阵。在空间 同步时，利用所确定的空间转换矩阵即可将多个探测传感器的数据统一至相同的坐标系中。
接下来，对本申请实施例涉及的执行主体进行简单介绍。
本申请实施例提供的多传感器信号融合的方法可以由电子设备来执行。该电子设备可以配置或连接有多个探测传感器，多个探测传感器的工作频段不相同。作为本申请的一个示例，多个探测传感器包括激光雷达传感器、毫米波雷达传感器。示例性地，激光雷达传感器可以包括但不限于8线激光雷达、16线激光雷达、24线激光雷达、32线激光雷达、64线激光雷达、128线激光雷达中的任一种。毫米波雷达传感器可以包括但不限于77GHz毫米波雷达、24GHz毫米波雷达中的任一种。
在实施中，激光雷达传感器和毫米波雷达传感器可以根据实际需求进行安装，譬如可以固定在交通路侧的标杆(横杆或立杆)上，以分别通过激光雷达传感器和毫米波雷达传感器对待检测的目标进行探测。作为一种示例，待检测的目标可以包括但不限于车辆、行人、非机动车辆、树木。
毫米波雷达传感器可以采用面阵的天线阵列，其覆盖的视场是前向180°空间或者更小。对于激光雷达传感器来说，包括有限的前向视场和360°环视视场。为了保证毫米波雷达传感器和激光雷达传感器的视场匹配，在一个示例中，在安装时，前向扫描的激光雷达传感器配合一个具有面阵的天线阵列的毫米波雷达传感器就可以完成对前向视场的扫描成像。对于360°环视视场的激光雷达传感器，则按需搭配3-4个具有面阵的天线阵列的毫米波雷达传感器，其中多个毫米波雷达传感器的视场存在部分重叠。或者，对于360°环视视场的激光雷达传感器，配合一个同样是360°环视视场的毫米波雷达传感器。
在一些实施例中，电子设备可以包括但不限于可穿戴设备、终端设备、车机设备、摄像设备、路侧基站。示例性地，可穿戴设备可以包括但不限于智能手表、智能手环、智能耳罩。另外，终端设备可以包括但不限于手机、平板电脑、增强现实(augmented reality，AR)/虚拟现实(virtual reality，VR)设备、笔记本电脑、超级移动个人计算机(ultra-mobile personal computer，UMPC)、上网本、个人数字助理(personal digital assistant，PDA)。
其中，路侧基站是智能交通车路协同的重要基础设施，是集感知、计算、通讯能力为一体的服务站。在一实施例中，路侧基站又可以称为智慧基站或路侧融合感知系统。
在介绍完本申请实施例涉及的术语和执行主体后，接下来结合附图对本申请实施例提供的多传感器信号融合的方法进行详细介绍。
请参考图1，图1是根据一示例性实施例示出的一种多传感器信号融合的方法流程图。作为示例而非限定，该方法可以应用于上述电子设备中。该方法可以包括如下几个步骤：
步骤101：获取同一时刻的、针对同一探测范围的多个探测传感器中每个探测传感器的采样信号，得到多个传感器采样信号，多个探测传感器的工作频段不相同。
在一个实施例中，多个探测传感器包括激光雷达传感器和毫米波雷达传感器。在另一个实施例中，多个探测传感器包括毫米波雷达传感器和可见光探测传感器。在又一个实施例中，多个探测传感器包括毫米波雷达传感器和红外探测传感器。
作为本申请的一个示例，电子设备通过配置的多个探测传感器进行目标探测。在探测过程中，电子设备对多个探测传感器中每个探测传感器的回波进行采样处理后，得到多个传感器采样信号。
作为本申请的一个示例，在通过多个探测传感器进行目标探测时，还可以对多个探测传感器进行时间同步和空间同步。
其中，时间同步是指通过多个传感器中的一个探测传感器触发多个传感器中的其他探测传感器工作。
作为本申请的一个示例，以多个探测传感器包括激光雷达传感器和毫米波雷达传感器为例，时间同步的具体实现可以包括：在激光雷达传感器转动扫描的过程中，每当激光雷达传感器的电机转过自身的零点时，向毫米波雷达传感器发送一个同步触发信号。毫米波雷达传感器在接收到该同步触发信号时，开始发送连续的电磁波。与此同时毫米波雷达传感器的接收天线开始接收电磁波的回波，在发送结束之后，回波的接收也随之结束，而激光雷达传感器也转动扫描过了相同的视场。如此，使得二者的采样信号在时间上同步。
需要说明的是，上述时间同步是以激光雷达传感器触发毫米波雷达传感器为例进行说明，在另一实施例中，时间同步还可以由毫米波雷达传感器触发激光雷达传感器，示例性地，毫米波雷达传感器每次在发送连续的电磁波的同时，向激光雷达传感器发送一个同步触发信号。激光雷达传感器接收同步触发信号后，开始执行探测操作。
空间同步是指通过多个探测传感器对相同测试目标的多次探测确定空间转换矩阵。示例性地，在获取时间上同步的激光采样信号和毫米波采样信号之后，可以基于空间转换矩阵对激光采样信号和毫米波采样信号进行空间同步处理，之后，基于空间同步处理后的激光采样信号和毫米波采样信号执行后续操作。
在实施中，由于各个探测传感器的频段不相同，他们的反射率不相同，所以，多个传感器采样信号的信号幅值差异可能较大。为此，可以分别对各个探测传感器的传感器采样信号进行正则化处理，以将各个传感器采样信号的信号幅值转化至统一的幅值范围内。
在一个实施例中，可以通过如下公式(4)分别对各个传感器采样信号进行正则化处理：
Figure PCTCN2022127075-appb-000007
其中，V norm表示正则化后的信号幅值，V表示原始的信号幅值，V min表示一帧探测数据中的最小信号幅值，V max表示一帧探测数据中的最大信号幅值。
之后，电子设备基于时间同步、空间同步、以及正则化处理后的多个传感器采样信号执行后续步骤。
步骤102：分别将多个传感器采样信号中的每个传感器采样信号同步至同一特征维度空间，得到多个信号数据，其中，特征维度空间是根据每个探测传感器的探测维度和系统输出要求设置的。
探测传感器的探测维度根据探测传感器自身的特性决定的，譬如，若探测传感器为激光雷达传感器，则激光雷达传感器的探测维度(这里称为第一特征维度空间)包括的特征维度为帧量特征、扫描光束特征、时间特征、幅值特征。其中，帧量特征是指激光采样信号对应多个数据帧，扫描光束特征是指激光采样信号是通过扫描光束扫描得到的，时间特征是指激光采样信号对应有采样时间，幅值特征是指激光采样信号是具有一定幅值的。
再如，若探测传感器为毫米波雷达传感器，则毫米波雷达传感器的探测维度(这里称为第二特征维度空间)包括的特征维度为啁啾特征、辐射天线特征、时间特征、幅值特征。其中，啁啾特征是指脉冲瞬时频率随时间的变化而变化的特性，辐射天线特征是指毫米波采样信号是通过辐射天线发射电磁波得到，时间特征是指毫米波采样信号是对应有采样时间，幅值特征是指毫米波采样信号是具有一定幅值的。
系统输出要求可以根据实际需求进行设置。
作为本申请的一个示例，譬如系统输出要求包括速度、角度和距离，则该特征维度空间包括速度特征、角度特征和距离特征。此时，电子设备将激光雷达传感器的采样信号从第一特征维度空间转换至该特征维度空间，以及将毫米波雷达传感器的采样信号从第二特征维度空间转换至该特征维度空间。
步骤103：对多个信号数据进行融合处理，得到空间信号描述矩阵，空间信号描述矩阵能够描述目标的信号回波质量。
在一个实施例中，电子设备可以采用加权、互相关或者二元假设检验的方法对多个信号数据进行融合，得到空间信号描述矩阵。
在一个实施例中，考虑到离散数字采样的问题，多个探测传感器的空间采样密度可能不一样，在该种情况下，步骤103的具体实现包括：当多个探测传感器的采样步长不相同时，通过插值法对齐多个信号数据的采样步长，以将多个探测传感器的特征空间采样步长对齐。对经对齐处理后的多个信号数据进行融合处理，得到空间信号描述矩阵。
步骤104：基于空间信号描述矩阵，输出融合数据。
作为本申请的一个示例，对空间信号描述矩阵中融合后的信号进行峰值提取，根据峰值提取结果输出融合数据。譬如将峰值提取结果作为融合数据输出。
在本申请实施例中，获取同一时刻的、针对同一探测范围的多个探测传感器中每个探测传感器的采 样信号，得到多个传感器采样信号，多个探测传感器的工作频段不相同。分别将多个传感器采样信号中的每个传感器采样信号同步至同一特征维度空间，得到多个信号数据，其中特征维度空间是根据每个探测传感器的探测维度和系统输出要求设置的。对多个信号数据进行融合处理，得到空间信号描述矩阵，空间信号描述矩阵能够描述目标的信号回波质量。基于空间信号描述矩阵，输出融合数据。如此，基于探测传感器的原始的传感器采样信号进行数据融合，在提高探测的鲁棒性的同时，避免因基于点云数据或者基于目标检测结果融合导致出现漏检等情况，提高了最终生成的融合数据的准确度。
接下来以多个探测传感器包括激光雷达传感器和毫米波雷达传感器为例进行说明。请参考图2，图2是根据另一示例性实施例示出的多传感器信号融合的方法流程示意图，该方法可以应用于上述电子设备中。作为示例而非限定，该方法可以包括如下几个实现步骤：
步骤201：获取同一时刻的、针对同一探测范围的激光雷达传感器和毫米波雷达传感器的采样信号，得到激光采样信号和毫米波采样信号。
其实现可以参见上述图1所示实施例中的步骤101。
步骤202：分别将激光采样信号和毫米波采样信号同步至同一特征维度空间，得到第一激光信号数据和毫米波信号数据。
在一个示例中，将激光雷达传感器的采样信号从第一特征维度空间转换至特征维度空间，第一特征维度空间包括的特征维度为帧量特征、扫描光束特征、时间特征，特征维度空间包括的特征维度为速度特征、角度特征、距离特征。以及将毫米波雷达传感器的采样信号从第二特征维度空间转换至特征维度空间，第二特征维度空间包括的特征维度为啁啾特征、辐射天线特征、时间特征。
步骤203：分别基于多个信号数据中的每个信号数据，确定能够评价目标的信号回波质量的至少一个评价参数，得到多组评价参数集合，多组评价参数集合中的每组评价参数集合包括至少一个评价参数。
在本实施例中，多个信号数据包括第一激光信号数据和毫米波信号数据。
作为本申请的一个示例，步骤203的具体实现可以包括：基于毫米波信号数据，确定目标对应的毫米波波峰位置和毫米波回波能量，得到一组评价参数集合。基于第一激光信号数据，确定目标对应的回波前沿位置和激光波峰饱和度，得到另一组评价参数集合，回波前沿位置是指目标被激光雷达传感器初次探测到时激光雷达传感器探测出的位置，激光波峰饱和度用于指示目标为真实目标的置信度。
其中，目标的毫米波波峰位置是该目标出现概率最大的位置，也就是说，该目标在毫米波波峰位置出现的概率最大。从数学角度来说，毫米波波峰位置是该目标的位置无偏估计。
作为本申请的一个示例，可以基于毫米波信号数据，确定第一角度距离图，之后基于第一角度距离图，确定毫米波波峰位置。在一个示例中，第一角度距离图为毫米波采样信号的频谱图。示例性地，可以对毫米波采样信号进行3D-FFT(Fast Fourier Transformation，快速傅里叶变换)处理，得到第一角度距离图。第一角度距离图的横坐标为角度，纵坐标为距离，对应的是极坐标系下的空间采样信号数据。
需要说明的是，上述仅是以对毫米波采样信号进行3D-FFT处理以得到第一角度距离图为例进行说明。在另一实施例中，还可以对毫米波采样信号进行小波变换处理，以确定第一角度距离图。在又一个实施例中，也可以对毫米波采样信号进行子空间变换处理，以确定第一角度距离图。
作为本申请的一个示例，基于第一角度距离图，通过2D-CFAR(Two Dimensional-Constant False-Alarm Rate，二维恒虚警算法)查找目标的毫米波波峰，确定毫米波波峰对应的角度和距离，之后根据该角度和距离确定毫米波波峰位置。示例性地，可以根据该角度和距离通过如下公式(5)确定毫米波波峰位置：
x 1＝ρcosθ，y 1＝ρsinθ (5)
其中，x 1表示毫米波波峰位置的横坐标，y 1表示毫米波波峰位置的纵坐标，ρ表示距离，θ表示角度。
在确定毫米波回波能量时，电子设备基于第一角度距离图，对该目标的回波值进行积分，即可确定该目标的毫米波回波能量。
电子设备将毫米波波峰位置和毫米波回波能量作为一组评价参数集合。
作为本申请的一个示例，电子设备基于第一激光信号数据，确定第二角度距离图，之后基于第二角度距离图，确定目标对应的回波前沿位置和激光波峰饱和度。
作为本申请的一个示例，电子设备可以直接根据第一激光信号数据，构建出第二角度距离图。同理，第二角度距离图的横坐标为角度，纵坐标为距离。
示例性地，请参考图3，图3是根据一示例性实施例示出的一种角度距离图的示意图，其中，白色高亮区域是激光雷达传感器的探测数据。在纵坐标中数值20所在位置，从左至右的一条直线为毫米波雷达传感器的探测数据。
激光雷达传感器在激光扫描的过程中，当初次探测到目标时，电子设备根据目标的回波确定对应的位置，这里将该位置称为回波前沿位置。譬如请参考图3，假设目标为图3中的31，目标31的回波前沿位置如图3中的32所示。由于激光雷达传感器是采用TOF方法进行测距，因此，本申请实施例提取回波前沿位置，利用回波前沿位置作为目标的位置无偏估计。
作为本申请的一个示例，基于第二角度距离图，通过1D-CFAR(One Dimensional-Constant False-Alarm Rate，一维恒虚警算法)对激光雷达传感器每个角度的波形进行波峰提取，然后确定波峰的前沿位置，得到回波前沿位置。
其中，激光波峰饱和度用于指示激光回波的特征，可以作为激光回波质量的评价系数。
作为本申请的一个示例，基于第二角度距离图确定激光雷达传感器探测的目标的激光波峰饱和度的具体实现可以包括：基于第二角度距离图，获取目标对应的激光回波值和激光回波脉冲宽度。对激光回波值进行积分，得到激光回波能量。将激光回波能量与激光回波脉冲宽度相除后得到的数值作为目标对应的激光波峰饱和度。
在一个实施例中，基于第二角度距离图，通过动态阈值法，确定目标对应的激光回波脉冲宽度。
需要说明的是，上述基于第二角度距离图确定激光雷达传感器探测的目标的激光波峰饱和度的方法仅是示例性的，在另一实施例中，其具体实现还可以包括：基于第二角度距离图，获取目标对应的激光回波值和激光回波脉冲宽度。对激光回波值进行积分，得到激光回波能量。确定激光回波脉冲宽度下的理论激光回波能量。将激光回波能量与理论激光回波能量相除后得到的数值作为目标对应的激光波峰饱和度。
步骤204：基于多组评价参数集合进行信号融合处理，得到空间信号描述矩阵。
作为本申请的一个示例，步骤204的具体实现可以包括：基于毫米波波峰位置和毫米波回波能量，确定毫米波波峰位置的方差，得到第一方差，第一方差用于指示目标在毫米波波峰位置处的概率分布情况。基于回波前沿位置和激光波峰饱和度，确定回波前沿位置的方差，得到第二方差，第二方差用于指示目标在回波前沿位置处的概率分布情况。基于毫米波波峰位置、回波前沿位置、第一方差、第二方差，确定空间信号描述矩阵。
作为示例而非限定，基于毫米波波峰位置和毫米波回波能量，通过如下公式(6)确定毫米波波峰位置的方差：
Figure PCTCN2022127075-appb-000008
其中，σ 1为第一方差，μ 1为毫米波波峰位置，包括(x 1,y 1)，feature表示毫米波回波能量，A和B是可调参数，具体可以根据实际需求进行设置。
作为示例而非限定，基于回波前沿位置和激光波峰饱和度，通过如下公式(7)确定回波前沿位置的方差：
Figure PCTCN2022127075-appb-000009
其中，σ 2为第二方差，μ 2为回波前沿位置，包括(x 2,y 2)，power表示激光波峰饱和度。
作为示例而非限定，基于毫米波波峰位置、回波前沿位置、第一方差、第二方差，通过如下公式(1)至(3)，确定空间信号描述矩阵：
Figure PCTCN2022127075-appb-000010
Figure PCTCN2022127075-appb-000011
Figure PCTCN2022127075-appb-000012
其中，μ 12为联合位置，包括(x 12,y 12)，μ 1为毫米波波峰位置，包括(x 1,y 1)，μ 2为回波前沿位置，包括(x 2,y 2)，σ 1为第一方差，σ 2为第二方差，(x,y)为变量，σ 12为联合置信度，V为联合置信度分布。
电子设备基于V可以确定联合置信度分布图，之后对联合置信度分布图进行转化，得到空间信号描述矩阵，也即利用空间信号描述矩阵表示联合置信度分布图。
需要说明的是，上述步骤203至步骤204是对多个信号数据进行融合处理的一种示例性实现方式。
步骤205：基于空间信号描述矩阵，输出融合数据。
作为本申请的一个示例，步骤205的具体实现可以包括：从空间信号描述矩阵中，确定联合置信度大于置信度阈值的联合位置。将联合置信度大于置信度阈值的联合位置和对应的联合置信度作为融合数据输出。
其中，置信度阈值可以由用户根据实际需求进行设置，或者，也可以由电子设备默认设置，本申请实施例对此不作限定。
由于联合位置是毫米波雷达传感器和激光雷达传感器共同检测到的目标出现概率最大的位置，所以，当联合位置的联合置信度大于置信度阈值时，说明该联合位置是目标出现概率最大的位置的置信度较大，也即可信度较高，因此电子设备将该联合位置和其联合置信度作为融合数据输出。
需要说明的是，基于空间信号描述矩阵输出融合数据的具体实现仅是示例性的，在另一实施例中，输出的融合数据还可以包括目标的其他探测数据。在一个实施例中，步骤205的具体实现还可以包括从空间信号描述矩阵中，确定联合置信度大于置信度阈值的联合位置。将联合置信度大于置信度阈值的联合位置、对应的联合置信度、速度作为融合数据输出。该速度是指毫米波雷达传感器探测到的速度。也即，输出的点云数据还可以包括毫米波雷达传感器探测的速度。在另一个实施例中，还可以将空间信号描述矩阵中的每个联合位置和每个联合位置的联合置信度作为点云数据输出，后续在使用点云数据时，可以根据每个联合位置的联合置信度进行筛选。
在一个实施例中，为了便于用户查看毫米波雷达传感器和激光雷达传感器各自探测的目标的位置分布情况，还可以绘制出毫米波雷达传感器的信号的置信度图，以及绘制出激光雷达传感器的信号的置信度图。
示例性，可以通过如下公式(8)和公式(9)确定毫米波雷达传感器的信号的置信度分布：
Figure PCTCN2022127075-appb-000013
Figure PCTCN2022127075-appb-000014
其中，V 1表示毫米波雷达传感器的信号的置信度分布。
如此，基于上述得到的V 1即可绘制出毫米波雷达传感器的信号的置信度图。
示例性，可以通过如下公式(10)和公式(11)确定激光雷达传感器的信号的置信度分布：
Figure PCTCN2022127075-appb-000015
Figure PCTCN2022127075-appb-000016
其中，V 2表示激光雷达传感器的信号的置信度分布，μ 0是激光雷达传感器的测距距离。
如此，基于上述得到的V 2即可绘制出激光雷达传感器的信号的置信度图。
在本申请实施例中，获取同一时刻的、针对同一探测范围的毫米波采样信号和激光采样信号。将毫米波采样信号和激光采样信号同步至同一特征维度空间中。根据同步后的毫米波采样信号获取第一角度距离图，根据同步后的激光采样信号获取第二角度距离图。对第一角度距离图和第二角度距离图进行融合处理，得到空间信号描述矩阵，此时空间信号描述矩阵用于描述目标的联合位置和联合位置的联合置信度，联合位置是目标出现概率最大的位置。如此，基于原始的采样信号进行融合处理，相比于基于点云数据或检测结果的融合，可以增强模糊但真实的目标的采样信号，避免误筛选掉该类目标。且根据融合后得到的空间信号描述矩阵，输出融合数据，由于空间信号描述矩阵描述了目标的联合位置和联合位置的联合置信度，所以可以提高最终生成的融合数据的准确度。
接下来以多个探测传感器包括激光雷达传感器和毫米波雷达传感器为例进行说明。请参考图4，图4是根据另一示例性实施例示出的多传感器信号融合的方法流程示意图，该方法可以应用于上述电子设备中。作为示例而非限定，该方法可以包括如下几个实现步骤：
步骤401至步骤402可以参见上述步骤201至步骤202。
步骤403：基于多个信号数据进行位置关联。
在本实施例中，多个信号数据包括第一激光信号数据和毫米波信号数据。
为了确定第一激光信号数据和毫米波信号数据中哪些信号数据是同一个目标的，电子设备基于第一激光信号数据和毫米波信号数据进行位置关联处理。
在本申请的一个示例中，可以采用二分图匹配法进行位置关联。示例性地，二分图匹配法可以为匈牙利算法等，本申请实施例对此不作限定。
在一个实施例中，探测范围内除了存在动态目标之外，还会存在静态目标。然而，由于毫米波雷达传感器探测的动态目标的回波容易被静态目标的回波淹没，因此，作为本申请的一个示例，电子设备可以对传感器信号数据中的静态目标进行滤除处理，以保留动态目标的信号数据，以便后续对动态目标的信号数据进行后续融合处理。
在一个示例中，电子设备从第一激光信号数据中滤除激光雷达传感器探测的静态目标的信号数据，得到第一动态目标信号数据，以及从毫米波信号数据中滤除毫米波雷达传感器探测的静态目标的信号数据，得到第二动态目标信号数据。
也即是，为了对动态目标的信号数据进行融合，电子设备对毫米波雷达传感器的毫米波信号数据中的静态目标的信号数据进行滤除处理，以及对激光雷达传感器的第一激光信号数据中的静态目标的信号数据进行滤除处理。
在一个实施例中，毫米波信号数据中包括速度。相应的，从毫米波信号数据中滤除毫米波雷达传感器探测的静态目标的信号数据，得到第二动态目标信号数据的具体实现可以包括：从毫米波信号数据中确定速度小于速度阈值的信号数据。从毫米波信号数据中删除速度小于速度阈值的信号数据。将经过删除处理后得到的毫米波信号数据确定为第二动态目标信号数据。
其中，速度阈值可以由用户根据实际需求进行设置，或者，也可以由电子设备默认设置，本申请实施例对此不作限定。
不难理解，若某目标的速度小于速度阈值，则说明该目标可能未动或微动，该种情况下可以将该目标确定为静态目标。电子设备从毫米波信号数据中删除该目标的信号数据。按照该种方式，即可将毫米波信号数据中的静态目标的信号数据滤除掉，剩下的即为毫米波雷达传感器探测到的动态目标的信号数据，也即得到第二动态目标信号数据。
需要说明的是，这里是以将速度小于速度阈值的目标确定为静态目标为例进行说明，在另一实施例中，还可以将速度小于或等于速度阈值的目标确定为静态目标，本申请实施例对此不作限定。
在一个实施例中，从第一激光信号数据中滤除激光雷达传感器探测的静态目标的信号数据，得到第一动态目标信号数据的具体实现可以包括：获取第二激光信号数据，第二激光信号数据是与第一激光信号数据相邻的一帧激光信号数据。将第一激光信号数据与第二激光信号数据相减后得到的信号数据差值作为第一动态目标信号数据。
作为一种示例，第二激光信号数据可以为第一激光信号数据的上一帧激光信号数据。作为另一种示例，第二激光信号数据也可以为第一激光信号数据的下一帧激光信号数据。
不难理解，若某目标是静止目标，则将激光雷达传感器的相邻两帧的信号数据相减后，就可以将静态目标的信号数据滤除掉。因此，对第一激光信号数据进行相邻帧回波作差后，即可确定激光雷达传感器探测到的动态目标的信号数据，也即得到第一动态目标信号数据。
当然需要说明的是，这里仅是以将第一激光信号数据与相邻的一帧激光信号数据相减以确定第一动态目标信号数据为例进行说明。在另一实施例中，还可以获取与第一激光信号数据之间相隔预设数量帧的激光信号数据，得到第三激光信号数据。之后，将第三激光信号数据与第一激光信号数据相减以确定第一动态目标信号数据。其中，预设数量可以根据实际需求进行设置。示例性地，还可以获取第二激光信号数据的前一帧或后一帧激光信号数据，得到第三激光信号数据，譬如，若第二激光信号数据为第一激光信号数据的上一帧激光信号数据，则可以将第二激光信号数据的前一帧作为第三激光信号数据，又如，若第二激光信号数据为第一激光信号数据的下一帧信号数据，则可以将第二激光信号数据的后一帧作为第三激光信号数据。本申请实施例对此不作限定。
作为本申请的一个示例，基于多个信号数据进行位置关联的具体实现包括：基于第一动态目标信号数据和第二动态目标信号数据进行位置关联。关联方法可以采用二分图匹配法。
步骤404：根据位置关联结果，分别从多个信号数据中的每个信号数据中，确定目标的信号数据，得到目标的多个目标信号数据。
通过位置关联后，可以确定激光雷达传感器和毫米波雷达传感器同时探测到的目标，或者说可以确定激光雷达传感器和毫米波雷达传感器探测到的同一目标。之后，分别从每个传感器信号数据中获取关联上的目标的信号数据，得到目标的多个目标信号数据。。
关联上的目标的数量可能为一个，或者也可能为多个，当关联上的目标的数量为多个时，每个关联上的目标的信号数据均可以按照下述方法进行融合。
需要说明的是，若多个探测传感器包括毫米波雷达传感器和可见光探测传感器，则同样可以采样位置匹配的方式确定这两者探测的同一目标，之后，从毫米波雷达传感器的毫米波信号数据中确定该同一目标的信号数据，以及从可见光探测传感器的可见光图像中确定该同一目标的信号数据，得到目标的多个信号数据。另外，若多个探测传感器包括毫米波雷达传感器和红外探测传感器，则亦是如此，这里不再重复赘述。
步骤405：对多个目标信号数据进行融合处理，得到空间信号描述矩阵。
在一个实施例中，将多个目标信号数据按照一定的排列方式进行排列，得到空间信号描述矩阵。譬如将多个目标信号数据中属于第一动态目标信号数据的目标信号数据排于第一列，以及将多个目标信号数据中属于第二动态目标信号数据的目标信号数据排于第二列，得到空间信号描述矩阵。
步骤406：基于空间信号描述矩阵，输出融合数据。
作为本申请的一个示例，步骤406的具体实现可以包括：获取第一动态目标信号数据中目标的角度、以及获取第二动态目标信号数据中目标的距离。将角度和距离确定为融合数据。
由于毫米波雷达传感器的测角分辨率低，所以毫米波雷达传感器对于目标到达角的测量可以由激光雷达传感器来提供，因此请参考图5，在进行数据融合时，将激光雷达传感器探测的第一动态目标信号数据中的角度作为输出。而毫米波雷达传感器的测距信息较为准确，激光雷达传感器的测距会受到噪声影响，所以目标的测距信息由毫米波雷达传感器来提供，因此将毫米波雷达传传感器的第二动态目标信号数据中的距离作为输出，如此即可得到融合数据。
作为本申请的一个示例，输出第一激光信号数据中静态目标的距离和角度。也即对于静态目标，可以直接将激光雷达传感器的探测结果作为输出。
示例性地，请参考图3，图3是根据一示例性实施例示出的一种信号数据的示意图，其中，横坐标为角度，从-10度到10度，纵坐标为距离，从0米到80米。图3中的白色亮度(这里以白色为例，还可以是其他颜色，如黄色)区域为激光雷达传感器的信号数据，距离为20米左右、从左至右的一条线为毫米波雷达传感器的信号数据。也即毫米波雷达传感器测到了位于20m处的目标，但却不知道其具体来自哪个角度，但激光雷达传感器可以很清楚地测到该目标的轮廓来自于大约5度附近。因此位于5度和20米左右的目标的距离可以由毫米波雷达传感器的测距结果给出，由激光雷达传感器提供其为于5度左右具体从哪个角度到哪个角度。另外，没有被毫米波雷达传感器测到的目标是被滤除的静态目标，通过激光雷达传感器的探测结果即可提供距离和角度。如此，可以得到图6所示的点云图。
作为本申请的另一个示例，步骤406的具体实现还可以包括：获取第一动态目标信号数据中目标的角度、以及获取第二动态目标信号数据中目标的距离和速度。将目标的角度、距离、速度确定为融合数据。
在本申请实施例中，获取同一时刻的、针对同一探测范围的多个探测传感器中每个探测传感器的采样信号，得到多个传感器采样信号，多个探测传感器的工作频段不相同。分别将多个传感器采样信号同步至同一特征维度空间。得到多个信号数据。基于目标的多个信号数据，将多个信号数据进行融合，得到融合数据。如此，基于探测传感器的原始的传感器采样信号进行数据融合，在提高探测的鲁棒性的同时，避免因基于点云数据或者基于目标检测结果融合导致出现漏检等情况，提高了最终生成的融合数据的准确度。
需要说明的是，在多个探测传感器包括毫米波雷达传感器和可见光探测传感器的情况下，在基于目标的多个信号数据进行融合时，可以将毫米波雷达传感器探测的信号数据作为可见光探测传感器探测的信号数据中的附加通道，得到融合数据。示例性地，可以将毫米波雷达传感器的信号数据中的距离作为可见光图像中的一个附加通道，再如，将毫米波雷达传感器的信号数据中的距离和角度分别作为可见光图像中的两个附加通道。
同理，在多个探测传感器包括毫米波雷达传感器和红外探测传感器的情况下，在基于目标的多个信号数据进行融合时，可以将毫米波雷达传感器探测的信号数据作为红外探测传感器探测的信号数据中的附加通道，得到融合数据。示例性地，可以将毫米波雷达传感器的信号数据中的距离作为红外图像中的一个附加通道，再如，将毫米波雷达传感器的信号数据中的距离和角度分别作为红外图像中的两个附加通道。
应理解，上述实施例中各步骤的序号并不意味着执行顺序的先后，各过程的执行顺序应以其功能和内在逻辑确定，而不应对本申请实施例的实施过程构成任何限定。
基于上述各个实施例提供的多传感器信号融合方法，请参考图7，图7是根据一示例性实施例示出的一种多传感器信号融合的装置的结构示意图，该装置可以由软件、硬件或者两者的结合实现称为上述电子设备的部分或者全部。该装置可以包括：
获取模块710，用于获取同一时刻的、针对同一探测范围的多个探测传感器中每个探测传感器的采样信号，得到多个传感器采样信号，所述多个探测传感器的工作频段不相同；
同步模块720，用于分别将所述多个传感器采样信号中的每个传感器采样信号同步至同一特征维度空间，得到多个信号数据，其中，所述特征维度空间是根据所述每个探测传感器的探测维度和系统输出要求设置的；
融合模块730，用于对所述多个信号数据进行融合处理，得到空间信号描述矩阵，所述空间信号描 述矩阵能够描述目标的信号回波质量；
输出模块740，用于基于所述空间信号描述矩阵，输出融合数据。
作为本申请的一个示例，所述融合模块730用于：
分别基于所述多个信号数据中的每个信号数据，确定能够评价所述目标的信号回波质量的至少一个评价参数，得到多组评价参数集合，所述多组评价参数集合中的每组评价参数集合包括所述至少一个评价参数；
基于所述多组评价参数集合进行信号融合处理。
作为本申请的一个示例，所述多个信号数据包括第一激光信号数据和毫米波信号数据；所述融合模块730用于：
所述分别基于所述多个信号数据中的每个信号数据，确定能够评价所述目标的信号回波质量的至少一个评价参数，得到多组评价参数集合，包括：
基于所述毫米波信号数据，确定所述目标对应的毫米波波峰位置和毫米波回波能量，得到一组评价参数集合；
基于所述第一激光信号数据，确定所述目标对应的回波前沿位置和激光波峰饱和度，得到另一组评价参数集合，所述回波前沿位置是指所述目标被激光雷达传感器初次探测到时所述激光雷达传感器探测出的位置，所述激光波峰饱和度用于指示所述目标为真实目标的置信度。
作为本申请的一个示例，所述融合模块730用于：
基于所述毫米波波峰位置和所述毫米波回波能量，确定所述毫米波波峰位置的方差，得到第一方差，所述第一方差用于指示所述目标在所述毫米波波峰位置处的概率分布情况；
基于所述回波前沿位置和所述激光波峰饱和度，确定所述回波前沿位置的方差，得到第二方差，所述第二方差用于指示所述目标在所述回波前沿位置处的概率分布情况；
基于所述毫米波波峰位置、所述回波前沿位置、所述第一方差、所述第二方差，确定所述空间信号描述矩阵。
作为本申请的一个示例，所述融合模块730用于：
基于所述毫米波波峰位置、所述回波前沿位置、所述第一方差、所述第二方差，通过如下公式(1)至(3)，确定所述空间信号描述矩阵：
Figure PCTCN2022127075-appb-000017
Figure PCTCN2022127075-appb-000018
Figure PCTCN2022127075-appb-000019
其中，所述μ 12包括(x 12,y 12)，所述μ 1为所述毫米波波峰位置，包括(x 1,y 1)，所述μ 2为所述回波前沿位置，包括(x 2,y 2)，所述σ 1为所述第一方差，所述σ 2为所述第二方差，所述(x,y)为变量，所述σ 12为联合置信度，所述V为联合置信度分布。
作为本申请的一个示例，所述输出模块740用于：
从所述空间信号描述矩阵中，确定联合置信度大于置信度阈值的联合位置；
将所述联合置信度大于置信度阈值的联合位置和对应的联合置信度作为所述融合数据输出。
作为本申请的一个示例，所述融合模块730用于：
基于所述多个信号数据进行位置关联；
根据位置关联结果，分别从所述多个信号数据中的每个信号数据中，确定所述目标的信号数据，得 到所述目标的多个目标信号数据；
基于所述多个目标信号数据进行融合处理。
作为本申请的一个示例，所述多个探测传感器包括激光雷达传感器和毫米波雷达传感器，所述多个信号数据包括第一激光信号数据和毫米波信号数据；
所述融合模块730用于：从所述第一激光信号数据中滤除所述激光雷达传感器探测的静态目标的信号数据，得到第一动态目标信号数据，以及从所述毫米波信号数据中滤除所述毫米波雷达传感器探测的静态目标的信号数据，得到第二动态目标信号数据；
相应的，所述融合模块730基于所述多个信号数据进行位置关联的具体实现包括：基于所述第一动态目标信号数据和所述第二动态目标信号数据进行位置关联。
作为本申请的一个示例，所述毫米波信号数据中包括速度，所述融合模块730用于：
从所述毫米波信号数据中删除所述速度小于所述速度阈值的信号数据；
将经过删除处理后得到的毫米波信号数据确定为所述第二动态目标信号数据。
作为本申请的一个示例，所述融合模块730用于：
获取第二激光信号数据，所述第二激光信号数据是与所述第一激光信号数据相邻的一帧激光信号数据；
将所述第一激光信号数据与所述第二激光信号数据相减后得到的信号数据差值作为所述第二动态目标信号数据。
作为本申请的一个示例，所述特征维度空间包括角度特征和距离特征，所述输出模块740用于：
获取所述第一动态目标信号数据中所述目标的角度、以及获取所述第二动态目标信号数据中所述目标的距离；
将所述角度和所述距离确定为所述融合数据。
作为本申请的一个示例，所述输出模块740还用于：
输出所述第一激光信号数据中所述静态目标的距离和角度。
作为本申请的一个示例，所述同步模块720还用于：
分别对所述多个传感器采样信号中的每个传感器采样信号进行正则化处理；
分别将经正则化处理后的所述每个传感器采样信号同步至所述特征维度空间。
作为本申请的一个示例，所述同步模块720还用于：
对所述多个探测传感器进行时间同步和空间同步，其中，所述时间同步是指通过所述多个传感器中的一个探测传感器触发所述多个传感器中的其他探测传感器工作，空间同步是指通过所述多个探测传感器对相同测试目标的多次探测确定空间转换矩阵。
作为本申请的一个示例，所述融合模块730用于：
当所述多个探测传感器的采样步长不相同时，通过插值法对齐所述多个信号数据的采样步长；
对经对齐处理后的所述多个信号数据进行融合处理。
作为本申请的一个示例，所述多个探测传感器包括激光雷达传感器和毫米波雷达传感器；
所述同步模块720用于：
将所述激光雷达传感器的采样信号从第一特征维度空间转换至所述特征维度空间，所述第一特征维度空间包括的特征维度为帧量特征、扫描光束特征、时间特征，所述特征维度空间包括的特征维度为速度特征、角度特征、距离特征；
将所述毫米波雷达传感器的采样信号从第二特征维度空间转换至所述特征维度空间，所述第二特征维度空间包括的特征维度为啁啾特征、辐射天线特征、时间特征。
在本申请实施例中，获取同一时刻的、针对同一探测范围的多个探测传感器中每个探测传感器的采样信号，得到多个传感器采样信号，多个探测传感器的工作频段不相同。分别将多个传感器采样信号中的每个传感器采样信号同步至同一特征维度空间，得到多个信号数据，其中特征维度空间是根据每个探测传感器的探测维度和系统输出要求设置的。对多个信号数据进行融合处理，得到空间信号描述矩阵，空间信号描述矩阵能够描述目标的信号回波质量。基于空间信号描述矩阵，输出融合数据。如此，基于探测传感器的原始的传感器采样信号进行数据融合，在提高探测的鲁棒性的同时，避免因基于点云数据 或者基于目标检测结果融合导致出现漏检等情况，提高了最终生成的融合数据的准确度。
图8为本申请一实施例提供的电子设备的结构示意图。如图8所示，该实施例的电子设备8包括：至少一个处理器80(图8中仅示出一个)、存储器81以及存储在所述存储器81中并可在所述至少一个处理器80上运行的计算机程序82，所述处理器80执行所述计算机程序82时实现上述任意各个方法实施例中的步骤。
所述电子设备8可以是桌上型计算机、笔记本、掌上电脑及云端服务器等计算设备。该电子设备可包括，但不仅限于，处理器80、存储器81。本领域技术人员可以理解，图8仅仅是电子设备8的举例，并不构成对电子设备8的限定，可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件，例如还可以包括输入输出设备、网络接入设备等。
所称处理器80可以是CPU(Central Processing Unit，中央处理单元)，该处理器80还可以是其他通用处理器、DSP(Digital Signal Processor，数字信号处理器)、ASIC(Application Specific Integrated Circuit，专用集成电路)、FPGA(Field-Programmable Gate Array，现成可编程门阵列)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。
所述存储器81在一些实施例中可以是所述电子设备8的内部存储单元，例如电子设备8的硬盘或内存。所述存储器81在另一些实施例中也可以是所述电子设备8的外部存储设备，例如所述电子设备8上配备的插接式硬盘，SMC(Smart Media Card,智能存储卡)，SD(Secure Digital,安全数字)卡，闪存卡(Flash Card)等。进一步地，所述存储器81还可以既包括所述电子设备8的内部存储单元也包括外部存储设备。所述存储器81用于存储操作系统、应用程序、引导装载程序(BootLoader)、数据以及其他程序等，例如所述计算机程序的程序代码等。所述存储器81还可以用于暂时地存储已经输出或者将要输出的数据。
需要说明的是，上述装置/单元之间的信息交互、执行过程等内容，由于与本申请方法实施例基于同一构思，其具体功能及带来的技术效果，具体可参见方法实施例部分，此处不再赘述。
所属领域的技术人员可以清楚地了解到，为了描述的方便和简洁，仅以上述各功能单元、模块的划分进行举例说明，实际应用中，可以根据需要而将上述功能分配由不同的功能单元、模块完成，即将所述装置的内部结构划分成不同的功能单元或模块，以完成以上描述的全部或者部分功能。实施例中的各功能单元、模块可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中，上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。另外，各功能单元、模块的具体名称也只是为了便于相互区分，并不用于限制本申请的保护范围。上述系统中单元、模块的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
以上所述实施例仅用以说明本申请的技术方案，而非对其限制；尽管参照前述实施例对本申请进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本申请各实施例技术方案的精神和范围，均应包含在本申请的保护范围之内。

Claims (19)

  1. 一种多传感器信号融合的方法，其特征在于，所述方法包括：
    获取同一时刻的、针对同一探测范围的多个探测传感器中每个探测传感器的采样信号，得到多个传感器采样信号，所述多个探测传感器的工作频段不相同；
    分别将所述多个传感器采样信号中的每个传感器采样信号同步至同一特征维度空间，得到多个信号数据，其中，所述特征维度空间是根据所述每个探测传感器的探测维度和系统输出要求设置的；
    对所述多个信号数据进行融合处理，得到空间信号描述矩阵，所述空间信号描述矩阵能够描述目标的信号回波质量；
    基于所述空间信号描述矩阵，输出融合数据。
  2. 如权利要求1所述的方法，其特征在于，所述对所述多个信号数据进行融合处理，包括：
    分别基于所述多个信号数据中的每个信号数据，确定能够评价所述目标的信号回波质量的至少一个评价参数，得到多组评价参数集合，所述多组评价参数集合中的每组评价参数集合包括所述至少一个评价参数；
    基于所述多组评价参数集合进行信号融合处理。
  3. 如权利要求2所述的方法，其特征在于，所述多个信号数据包括第一激光信号数据和毫米波信号数据；
    所述分别基于所述多个信号数据中的每个信号数据，确定能够评价所述目标的信号回波质量的至少一个评价参数，得到多组评价参数集合，包括：
    基于所述毫米波信号数据，确定所述目标对应的毫米波波峰位置和毫米波回波能量，得到一组评价参数集合；
    基于所述第一激光信号数据，确定所述目标对应的回波前沿位置和激光波峰饱和度，得到另一组评价参数集合，所述回波前沿位置是指所述目标被激光雷达传感器初次探测到时所述激光雷达传感器探测出的位置，所述激光波峰饱和度用于指示所述目标为真实目标的置信度。
  4. 如权利要求3所述的方法，其特征在于，所述基于所述多组评价参数集合进行信号融合处理，包括：
    基于所述毫米波波峰位置和所述毫米波回波能量，确定所述毫米波波峰位置的方差，得到第一方差，所述第一方差用于指示所述目标在所述毫米波波峰位置处的概率分布情况；
    基于所述回波前沿位置和所述激光波峰饱和度，确定所述回波前沿位置的方差，得到第二方差，所述第二方差用于指示所述目标在所述回波前沿位置处的概率分布情况；
    基于所述毫米波波峰位置、所述回波前沿位置、所述第一方差、所述第二方差，确定所述空间信号描述矩阵。
  5. 如权利要求4所述的方法，其特征在于，所述基于所述毫米波波峰位置、所述回波前沿位置、所述第一方差、所述第二方差，确定所述空间信号描述矩阵，包括：
    基于所述毫米波波峰位置、所述回波前沿位置、所述第一方差、所述第二方差，通过如下公式(1)至(3)，确定所述空间信号描述矩阵：
    Figure PCTCN2022127075-appb-100001
    Figure PCTCN2022127075-appb-100002
    Figure PCTCN2022127075-appb-100003
    其中，所述μ 12包括(x 12,y 12)，所述μ 1为所述毫米波波峰位置，包括(x 1,y 1)，所述μ 2为所述回波前沿位置，包括(x 2,y 2)，所述σ 1为所述第一方差，所述σ 2为所述第二方差，所述(x,y)为变量，所述σ 12为联合置信度，所述V为联合置信度分布。
  6. 如权利要求3-5中任一项所述的方法，其特征在于，所述基于所述空间信号描述矩阵，输出融合数据，包括：
    从所述空间信号描述矩阵中，确定联合置信度大于置信度阈值的联合位置；
    将所述联合置信度大于置信度阈值的联合位置和对应的联合置信度作为所述融合数据输出。
  7. 如权利要求1所述的方法，其特征在于，所述对所述多个信号数据进行融合处理，包括：
    基于所述多个信号数据进行位置关联；
    根据位置关联结果，分别从所述多个信号数据中的每个信号数据中，确定所述目标的信号数据，得到所述目标的多个目标信号数据；
    基于所述多个目标信号数据进行融合处理。
  8. 如权利要求7所述的方法，其特征在于，所述多个探测传感器包括激光雷达传感器和毫米波雷达传感器，所述多个信号数据包括第一激光信号数据和毫米波信号数据；
    所述基于所述多个信号数据进行位置关联之前，还包括：
    从所述第一激光信号数据中滤除所述激光雷达传感器探测的静态目标的信号数据，得到第一动态目标信号数据，以及从所述毫米波信号数据中滤除所述毫米波雷达传感器探测的静态目标的信号数据，得到第二动态目标信号数据；
    所述基于所述多个信号数据进行位置关联，包括：
    基于所述第一动态目标信号数据和所述第二动态目标信号数据进行位置关联。
  9. 如权利要求8所述的方法，其特征在于，所述毫米波信号数据中包括速度，所述从所述毫米波信号数据中滤除所述毫米波雷达传感器探测的静态目标的信号数据，得到第二动态目标信号数据，包括：
    从所述毫米波信号数据中确定速度小于速度阈值的信号数据；
    从所述毫米波信号数据中删除所述速度小于所述速度阈值的信号数据；
    将经过删除处理后得到的毫米波信号数据确定为所述第二动态目标信号数据。
  10. 如权利要求8所述的方法，其特征在于，所述从所述第一激光信号数据中滤除所述激光雷达传感器探测的静态目标的信号数据，得到第一动态目标信号数据，包括：
    获取第二激光信号数据，所述第二激光信号数据是与所述第一激光信号数据相邻的一帧激光信号数据；
    将所述第一激光信号数据与所述第二激光信号数据相减后得到的信号数据差值作为所述第一动态目标信号数据。
  11. 如权利要求8所述的方法，其特征在于，所述特征维度空间包括角度特征和距离特征，所述基于所述空间信号描述矩阵，输出融合数据，包括：
    获取所述第一动态目标信号数据中所述目标的角度、以及获取所述第二动态目标信号数据中所述目标的距离；
    将所述角度和所述距离确定为所述融合数据。
  12. 如权利要求8所述的方法，其特征在于，所述方法还包括：
    输出所述第一激光信号数据中所述静态目标的距离和角度。
  13. 如权利要求1所述的方法，其特征在于，所述分别将所述多个传感器采样信号中的每个传感器采样信号同步至同一特征维度空间之前，还包括：
    分别对所述多个传感器采样信号中的每个传感器采样信号进行正则化处理；
    所述分别将所述多个传感器采样信号中的每个传感器采样信号同步至同一特征维度空间，包括：
    分别将经正则化处理后的所述每个传感器采样信号同步至所述特征维度空间。
  14. 如权利要求1所述的方法，其特征在于，所述方法还包括：
    对所述多个探测传感器进行时间同步和空间同步，其中，所述时间同步是指通过所述多个传感器中的一个探测传感器触发所述多个传感器中的其他探测传感器工作，空间同步是指通过所述多个探测传感器对相同测试目标的多次探测确定空间转换矩阵。
  15. 如权利要求1所述的方法，其特征在于，所述对所述多个信号数据进行融合处理，包括：
    当所述多个探测传感器的采样步长不相同时，通过插值法对齐所述多个信号数据的采样步长；
    对经对齐处理后的所述多个信号数据进行融合处理。
  16. 如权利要求1所述的方法，其特征在于，所述多个探测传感器包括激光雷达传感器和毫米波雷达传感器；
    所述分别将所述多个传感器采样信号中的每个传感器采样信号同步至同一特征维度空间，包括：
    将所述激光雷达传感器的采样信号从第一特征维度空间转换至所述特征维度空间，所述第一特征维度空间包括的特征维度为帧量特征、扫描光束特征、时间特征，所述特征维度空间包括的特征维度为速度特征、角度特征、距离特征；
    将所述毫米波雷达传感器的采样信号从第二特征维度空间转换至所述特征维度空间，所述第二特征维度空间包括的特征维度为啁啾特征、辐射天线特征、时间特征。
  17. 一种多传感器信号融合的装置，其特征在于，所述装置包括：
    获取模块，用于获取同一时刻的、针对同一探测范围的多个探测传感器中每个探测传感器的采样信号，得到多个传感器采样信号，所述多个探测传感器的工作频段不相同；
    同步模块，用于分别将所述多个传感器采样信号中的每个传感器采样信号同步至同一特征维度空间，得到多个信号数据，其中，所述特征维度空间是根据所述每个探测传感器的探测维度和系统输出要求设置的；
    融合模块，用于对所述多个信号数据进行融合处理，得到空间信号描述矩阵，所述空间信号描述矩阵能够描述目标的信号回波质量；
    输出模块，用于基于所述空间信号描述矩阵，输出融合数据。
  18. 一种电子设备，包括存储器、处理器以及存储在所述存储器中并可在所述处理器上运行的计算机程序，其特征在于，所述处理器执行所述计算机程序时实现如权利要求1至16任一项所述的方法。
  19. 一种计算机可读存储介质，所述计算机可读存储介质上存储有指令，其特征在于，所述指令被处理器执行时实现权利要求1至16任一项所述的方法。
PCT/CN2022/127075 2021-10-26 2022-10-24 多传感器信号融合的方法、装置、电子设备及存储介质 WO2023071992A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111251503.2A CN116027318A (zh) 2021-10-26 2021-10-26 多传感器信号融合的方法、装置、电子设备及存储介质
CN202111251503.2 2021-10-26

Publications (1)

Publication Number Publication Date
WO2023071992A1 true WO2023071992A1 (zh) 2023-05-04

Family

ID=86069286

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/127075 WO2023071992A1 (zh) 2021-10-26 2022-10-24 多传感器信号融合的方法、装置、电子设备及存储介质

Country Status (2)

Country Link
CN (1) CN116027318A (zh)
WO (1) WO2023071992A1 (zh)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116381674A (zh) * 2023-06-02 2023-07-04 陕西欧卡电子智能科技有限公司 一种水面激光雷达点云与毫米波雷达点云的融合方法
CN117031442A (zh) * 2023-10-08 2023-11-10 中国地质大学（武汉） 融合多通道波形数据的激光雷达水域地形测量方法及装置
CN117452407A (zh) * 2023-12-26 2024-01-26 哈尔滨工业大学(深圳)(哈尔滨工业大学深圳科技创新研究院) 一种用于车载辅助驾驶系统的雷达数据服务系统和方法
CN117968860A (zh) * 2024-03-13 2024-05-03 苏州优米康通信技术有限公司 一种多传感融合的障碍物检测方法

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090212995A1 (en) * 2007-12-20 2009-08-27 Shunguang Wu Distributed iterative multimodal sensor fusion method for improved collaborative localization and navigation
CN105353355A (zh) * 2015-11-16 2016-02-24 乐山师范学院 一种基于稀疏重构和投影成像的多基地雷达多目标定位方法
CN106405555A (zh) * 2016-09-23 2017-02-15 百度在线网络技术（北京）有限公司 用于车载雷达系统的障碍物检测方法和装置
CN109061643A (zh) * 2018-08-13 2018-12-21 南京理工大学 基于三维属性散射中心模型的多雷达融合高分辨成像方法
CN109343051A (zh) * 2018-11-15 2019-02-15 众泰新能源汽车有限公司 一种用于高级辅助驾驶的多传感器数据融合方法
US20190353774A1 (en) * 2018-05-18 2019-11-21 Industrial Technology Research Institute Object detection system, autonomous vehicle using the same, and object detection method thereof
CN112363167A (zh) * 2020-11-02 2021-02-12 重庆邮电大学 一种基于毫米波雷达与单目相机融合的扩展目标跟踪方法

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090212995A1 (en) * 2007-12-20 2009-08-27 Shunguang Wu Distributed iterative multimodal sensor fusion method for improved collaborative localization and navigation
CN105353355A (zh) * 2015-11-16 2016-02-24 乐山师范学院 一种基于稀疏重构和投影成像的多基地雷达多目标定位方法
CN106405555A (zh) * 2016-09-23 2017-02-15 百度在线网络技术（北京）有限公司 用于车载雷达系统的障碍物检测方法和装置
US20190353774A1 (en) * 2018-05-18 2019-11-21 Industrial Technology Research Institute Object detection system, autonomous vehicle using the same, and object detection method thereof
CN109061643A (zh) * 2018-08-13 2018-12-21 南京理工大学 基于三维属性散射中心模型的多雷达融合高分辨成像方法
CN109343051A (zh) * 2018-11-15 2019-02-15 众泰新能源汽车有限公司 一种用于高级辅助驾驶的多传感器数据融合方法
CN112363167A (zh) * 2020-11-02 2021-02-12 重庆邮电大学 一种基于毫米波雷达与单目相机融合的扩展目标跟踪方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
CHEN, QIPING: "Sensing system of environmental perception technologies for driverless vehicle: A review of state of the art and challenges", SENSORS AND ACTUATORS A: PHYSICAL, vol. 319, no. 1, 20 January 2021 (2021-01-20), XP086490316, ISSN: 0924-4247, DOI: 10.1016/j.sna.2021.112566 *

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116381674A (zh) * 2023-06-02 2023-07-04 陕西欧卡电子智能科技有限公司 一种水面激光雷达点云与毫米波雷达点云的融合方法
CN116381674B (zh) * 2023-06-02 2023-08-22 陕西欧卡电子智能科技有限公司 一种水面激光雷达点云与毫米波雷达点云的融合方法
CN117031442A (zh) * 2023-10-08 2023-11-10 中国地质大学（武汉） 融合多通道波形数据的激光雷达水域地形测量方法及装置
CN117031442B (zh) * 2023-10-08 2024-01-02 中国地质大学（武汉） 融合多通道波形数据的激光雷达水域地形测量方法及装置
CN117452407A (zh) * 2023-12-26 2024-01-26 哈尔滨工业大学(深圳)(哈尔滨工业大学深圳科技创新研究院) 一种用于车载辅助驾驶系统的雷达数据服务系统和方法
CN117452407B (zh) * 2023-12-26 2024-03-08 哈尔滨工业大学(深圳)(哈尔滨工业大学深圳科技创新研究院) 一种用于车载辅助驾驶系统的雷达数据服务系统和方法
CN117968860A (zh) * 2024-03-13 2024-05-03 苏州优米康通信技术有限公司 一种多传感融合的障碍物检测方法

Also Published As

Publication number Publication date
CN116027318A (zh) 2023-04-28

Similar Documents

Publication Publication Date Title
WO2023071992A1 (zh) 多传感器信号融合的方法、装置、电子设备及存储介质
CN111650582B (zh) 一种基于mimo毫米波雷达的手势识别方法
US8718323B2 (en) Batch detection association for enhanced target descrimination in dense detection environments
CN112946630B (zh) 一种基于毫米波雷达的人员计数跟踪方法
KR20200144862A (ko) 레이더의 해상도 향상 방법 및 장치
CN113009442B (zh) 一种雷达静止反射面多径目标的识别方法及装置
WO2023124780A1 (zh) 点云数据增强方法、装置、计算机设备、系统及存储介质
CN111398943B (zh) 目标姿态的确定方法及终端设备
CN115061113B (zh) 用于雷达的目标检测模型训练方法、装置及存储介质
CN113537417B (zh) 基于雷达的目标识别方法、装置、电子设备及存储介质
CN111046877A (zh) 一种毫米波图像可疑物品检测方法和系统
CN108535714B (zh) 一种毫米波雷达检测开放空间遮挡物体的方法及装置
CN113009441A (zh) 一种雷达运动反射面多径目标的识别方法及装置
KR20170134092A (ko) 고해상도 거리 프로파일을 이용한 다중 물체 판단 장치 및 방법
CN110992398A (zh) 一种激光雷达与红外探测系统图像融合跟踪与抗干扰方法
KR20220141748A (ko) 레이더 신호로부터 표적 정보를 추출하는 방법 및 컴퓨터 판독가능 저장 매체
Prabhakara et al. Exploring mmWave Radar and Camera Fusion for High-Resolution and Long-Range Depth Imaging
CN116027288A (zh) 生成数据的方法、装置、电子设备及存储介质
CN116466307B (zh) 一种基于深度图的毫米波雷达点云仿真的方法和装置
WO2023083164A1 (zh) 目标追踪方法、信号融合方法、装置、终端和存储介质
CN114492676B (zh) 一种目标检测模型的构建方法、装置和电子设备
WO2020133041A1 (zh) 车速计算方法、系统、设备及存储介质
CN112835025A (zh) 基于雷达检测的目标速度解模糊方法、装置及终端设备
Miftakhutdinov et al. A Technique of Object Detection Using Information about Changes in the Terrain Image
Wu et al. Deep learning-based uav detection in the low altitude clutter background

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22885869

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE