CN208937705U - A kind of device of multi-source heterogeneous sensor characteristics depth integration - Google Patents
A kind of device of multi-source heterogeneous sensor characteristics depth integration Download PDFInfo
- Publication number
- CN208937705U CN208937705U CN201821413848.7U CN201821413848U CN208937705U CN 208937705 U CN208937705 U CN 208937705U CN 201821413848 U CN201821413848 U CN 201821413848U CN 208937705 U CN208937705 U CN 208937705U
- Authority
- CN
- China
- Prior art keywords
- source heterogeneous
- heterogeneous sensor
- laser radar
- machine vision
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Abstract
The utility model provides a kind of device of multi-source heterogeneous sensor characteristics depth integration, is related to field of artificial intelligence.The device includes the multi-source heterogeneous sensor array, ZYNQ-7000 platform, FLASH large size storage chip of N number of laser radar and machine vision.The multidimensional data feature of target is obtained by multi-source heterogeneous multiple sensors, in ZYNQ-7000 platform, it is converted by the translation specifications of different multi-source heterogeneous sensors, form the feature space with unified dimensional, by establishing deep learning network, multi-source heterogeneous sensor is merged, realize machine vision and laser radar target acquistion, information extraction and tagsort and is merged.The utility model identifies for unmanned machine target signature and Decision Control provides environmental data accurate, reliable, with robustness, is of great significance to improving active collision avoidance identifying system performance, reducing collision accident.
Description
Technical field
The utility model relates to field of artificial intelligence more particularly to a kind of multi-source heterogeneous sensor characteristics depth integrations
Device.
Background technique
Currently, unmanned machine quantity continues to increase so that various accidents frequently occur, and causes the serious lives and properties to damage
It loses.The reason is that single-sensor used by unmanned machine, such as radar, vision, it can not completely obtain target information.Needle
The significant conditions such as motion state, distance state, location status to information can not carry out complete perception.
Utility model content
In view of the above-mentioned deficiencies of the prior art, technical problem to be solved by the utility model is to provide a kind of multi-source heterogeneous biographies
The device of sensor depths of features fusion is based on depth learning technology using ZYNQ-7000 platform, complete by deep learning model
At space, temporal characteristics information fusion algorithm, accurate, reliable, tool is provided for the identification of unmanned machine target signature and Decision Control
There is the environmental data of robustness, is of great significance to improving active collision avoidance identifying system performance, reducing collision accident.
In order to solve the above technical problems, technical solution adopted in the utility model is:
A kind of device of multi-source heterogeneous sensor characteristics depth integration, the multi-source including N number of laser radar and machine vision
Heterogeneous sensor array, ZYNQ-7000 platform, FLASH large size storage chip；ZYNQ-7000 platform includes ARM Cortex A9
Double-core, fpga logic computing unit；N number of laser radar and the multielement bar array of machine vision pass through optical fiber interface or net
Network interface and ARM Cortex A9 double-core, which are realized, to be connected；The external FLASH large size storage chip of fpga logic computing unit.
Further, the laser radar in the multi-source heterogeneous sensor array uses solid-state laser radar Leddar Vu,
Realize 8 Line beam quantity, it can be achieved that maximum detectable range be 185 meters；Machine view in the multi-source heterogeneous sensor array
Feel and DS-2CD3T25-I3 is regarded using infrared high-definition network camera Haikang prestige.
The beneficial effects of adopting the technical scheme are that multi-source heterogeneous sensor provided by the utility model is special
The device of sign depth integration is obtained by being installed on the multi-source heterogeneous multiple sensors of the equipments such as unmanned plane, automobile, dirigible, satellite
Environmental information is taken, target range, contour feature and machine vision are obtained by laser radar and obtain object pixel feature, is led to
It crosses ZYNQ-7000 platform and establishes deep learning network, multi-source heterogeneous sensor is merged, realize machine vision and laser
Radar target capture, information extraction and tagsort and merge, for equipment make decisions on one's own provide it is accurate, reliable, there is robust
Property decision adjudicate performance.
Detailed description of the invention
Fig. 1 is the apparatus structure block diagram of multi-source heterogeneous sensor characteristics depth integration provided by the embodiment of the utility model；
Fig. 2 is the logic unit of the method for multi-source heterogeneous sensor characteristics depth integration provided by the embodiment of the utility model
Connection figure；
Fig. 3 is the machine vision provided by the embodiment of the utility model based on deep learning algorithm and laser fusion decision-making mode
Network schematic diagram.
Specific embodiment
With reference to the accompanying drawings and examples, specific embodiment of the present utility model is described in further detail.Below
Embodiment is not intended to limit the scope of the present invention for illustrating the utility model.
A kind of device of multi-source heterogeneous sensor characteristics depth integration, the multi-source including N number of laser radar and machine vision
Heterogeneous sensor array, ZYNQ-7000 platform, FLASH large size storage chip；ZYNQ-7000 platform is a full programmable chip
Upper system (All Programmable SoC), the integrated chip ARM CortexA9 double-core, fpga logic computing unit；N
A laser radar and the multielement bar array of machine vision are bis- by optical fiber interface or network interface and ARM CortexA9
Verify existing connection；The external FLASH large size storage chip of fpga logic computing unit.
Multi-source heterogeneous sensor array is used to obtain the multidimensional data feature of target, the point cloud data including laser radar,
And the pixel data of machine vision.
Fpga logic computing unit, for realizing machine vision and laser radar target acquistion, information extraction and feature point
Class and fusion, including target identification and categorization module, space coordinate transformation module, information Fusion Module, behaviour decision making module, mesh
Mark is not and categorization module is established for realizing target identification and feature storehouse matching, and space coordinate transformation module is for realizing sharp
Optical radar maps to form unified information plane by coordinate, and realizes between machine vision module coordinate system and pixel coordinate system
Transformational relation, for information Fusion Module for establishing deep neural network, building radar captures target signature Space integration model,
Multiple Source Sensor carries out feature correction simultaneously, multi-sensor data Fusion in Time model is constructed, by multi-source heterogeneous sensing data
It is merged, and is corrected by spatial model；Behaviour decision making module is used to calculate spy using time and Space integration model
Subpoint of the target on feature space is surveyed, while the region of interest comprising machine vision target image will be established in subpoint
Domain scans for the detection for completing target identification object using deep neural network in area-of-interest, and when determining front
In the case where barrier, by the movement state information of the barrier submit to the central control system based on deep learning network into
Row intelligent collision warning decision.
In the present embodiment, the laser radar in multi-source heterogeneous sensor array uses solid-state laser radar Leddar Vu, real
Existing 8 Line beam quantity, it can be achieved that maximum detectable range be 185 meters.Machine vision in multi-source heterogeneous sensor array uses
Infrared high-definition network camera Haikang prestige regards DS-2CD3T25-I3.
Multi-source heterogeneous sensor characteristics depth is realized using the device of above-mentioned multi-source heterogeneous sensor characteristics depth integration
The method of fusion, this method are specific as follows:
Environmental information is obtained by multi-source heterogeneous multiple sensors, that is, obtains the multidimensional data feature of target, especially by
Laser radar obtains target range, contour feature, these data characteristicses are located in radar two-dimensional scanning plane coordinate system, passes through thunder
Up to two-dimensional imaging space, target obstacle relative tertiary location is provided；Object pixel feature is obtained by machine vision；
According to laser radar and machine vision fixed position relative, the priori knowledge established by distance passes through Bayes
The prior model of network, forms training deep learning network, and the deep learning network completed by training establishes laser radar seat
Transformational relation between mark system and machine vision coordinate system, and then construct radar and capture target signature Space integration model；By machine
The two dimension identification target pixel points of vision, are mapped in radar two-dimensional surface, are known according to computer machine visual environment correlation
Other algorithm and machine vision imaging principle are established between machine vision coordinate system and pixel coordinate system by depth integration network
Linear transformation relationship；Simultaneously in view of machine vision is to the distortion phenomenon of target imaging, pass through the machine vision training library of priori
Carry out nonlinear distortion correction；In conjunction with both the above transformational relation, laser radar coordinate system and machine vision image pixel are realized
Conversion between coordinate；
In conjunction with the Multiple Source Sensors such as acoustics, infrared, thermal imaging, GPS, the target signature for forming special scenes is obtained, is carried out
Feature correction constructs multi-sensor data Fusion in Time model, and is corrected by spatial model；
Subpoint of the detection target on feature space is calculated using time and Space integration model, while by subpoint
It is middle to establish the area-of-interest comprising machine vision target image, it is scanned in area-of-interest using deep learning network
Complete the detection of target identification object；In the case where determining front obstacle, the movement state information of the barrier is submitted
Intelligent collision warning decision is carried out to the central control system based on deep learning network.
Data characteristics depth integration method based on the Multiple Source Sensors such as radar, machine vision, acoustics, infrared, GPS is logical
Machine Machine Vision Detection authentication module is crossed, can be used for carrying out existence verifying to the target of laser radar primary election, exclusively
Face, sky etc. do not constitute dangerous obstacle target, distinguish lateral barrier etc..Machine machine vision applications are in environment measuring system
The accuracy and real-time of data processing must be taken into consideration in system.Method provided in this embodiment, depth integration process are by multi-source
Heterogeneous sensor data carry out message complementary sense and optimum organization processing on time, space, compensate for single-sensor measurement letter
Cease incomplete defect.It is overlapped according to different sensors in the two-dimensional imaging space of itself, passes through related priori knowledge, shape
At converged network.The training network for multi-source heterogeneous sensor fusion process it is accurate whether, directly affect radar data
With machine machine vision image data, including the system of acoustic sensor, infrared sensor and GPS signal on time, space
One property is related to final effective target correctness.When completion multi-sensor data Space integration.Need priori special by training
Library load correction is levied, so that the synchronization of multi-source heterogeneous sensor measurement data in time, spatially.
As shown in Figure 1, the logic unit of the method for multi-source heterogeneous sensor characteristics depth integration provided in this embodiment
Connection figure.Implementation is made of laser radar and machine vision two parts.Machine vision passes through Coordinate transformation systems and laser
Radar connection, maps to form unified information plane by coordinate.Machine vision passes through target identification and classification and target signature
Library, carries out target identification and feature storehouse matching is established.Laser radar is carried out by target identification and motion information trapping module
Target identification.Laser radar and machine vision carry out informix decision by data fusion module, and then complete multi-source and pass
Sensor fusion forms comprehensive anticollision.
As shown in Fig. 2, merging decision networks with laser radar for the machine machine vision based on deep learning algorithm, pass through
The comprehensive of the target signature data module of the target signature data module and machine vision of front end radar extracts, merges, and completes letter
The depth decision of breath.Front end data feature interface is made decisions and is shared by deep learning network.
The working principle of this method is as follows:
First against forward direction, lateral obstacle position information, distance measurement is carried out by radar, acoustics.In order to obtain
Accurate distance information, system are obtained for self-position, itself used GPS sensor obtains in real time.It is smart when obtaining
After true location information, by machine vision sensor, target signature is obtained.Radar, acoustic sensor are used as front system most heavy
The information position obtaining widget wanted, for obtaining the movement state information of front environmental goals, robustness, the essence of work in real time
Whether the function that degree directly affects detection system is realized.The measurement environment of laser radar is complex in actual environment, radar number
It needs that the detection of obstacles under various operating conditions can be completed according to processing module, and provides reliable and stable, accurate, symbol to central control system
Close the obstacle information of actual conditions.To realize front obstacle detection function, need effectively to know from actual traffic environment
The target that Chu do not need to avoid, and obtain its accurate motion information.And then region of interest is established by machine vision sensor
The positioning in domain reduces image detection, pattern-recognition search range, avoids carrying out ergodic search in entire image, improve
The real-time of detection system.Existence verifying is carried out to barrier using image, increases the accuracy and robust of detection system
Property.In the case where determining front obstacle, the movement state information of the barrier is submitted to based on deep learning network
Central control system carries out intelligent collision warning decision.
The utility model completes space, temporal characteristics information fusion algorithm by deep learning model, is unmanned machine mesh
Mark feature identifies and Decision Control provides accurate, the reliable, environmental data with robustness, to raising active collision avoidance identifying system
Performance, reduction collision accident are of great significance.
Finally, it should be noted that above embodiments are only to illustrate the technical solution of the utility model, rather than its limitations；
Although the utility model is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that:
It can still modify to technical solution documented by previous embodiment, or to some or all of the technical features
It is equivalently replaced；And these are modified or replaceed, it does not separate the essence of the corresponding technical solution, and the utility model right is wanted
Seek limited range.
Claims (2)
1. a kind of device of multi-source heterogeneous sensor characteristics depth integration, it is characterised in that: regarded including N number of laser radar and machine
Multi-source heterogeneous sensor array, the ZYNQ-7000 platform, FLASH large size storage chip felt；ZYNQ-7000 platform includes ARM
Cortex A9 double-core, fpga logic computing unit；N number of laser radar and the multielement bar array of machine vision pass through optical fiber
Interface or network interface and ARM Cortex A9 double-core, which are realized, to be connected；The external FLASH large size storage of fpga logic computing unit
Chip.
2. the device of multi-source heterogeneous sensor characteristics depth integration according to claim 1, it is characterised in that: the multi-source
Laser radar in heterogeneous sensor array uses solid-state laser radar Leddar Vu, realize 8 Line beam quantity, it can be achieved that
Maximum detectable range is 185 meters；Machine vision in the multi-source heterogeneous sensor array uses infrared high-definition network camera
Haikang prestige regards DS-2CD3T25-I3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201821413848.7U CN208937705U (en) | 2018-08-30 | 2018-08-30 | A kind of device of multi-source heterogeneous sensor characteristics depth integration |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201821413848.7U CN208937705U (en) | 2018-08-30 | 2018-08-30 | A kind of device of multi-source heterogeneous sensor characteristics depth integration |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN208937705U true CN208937705U (en) | 2019-06-04 |
Family
ID=66719314
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201821413848.7U Expired - Fee Related CN208937705U (en) | 2018-08-30 | 2018-08-30 | A kind of device of multi-source heterogeneous sensor characteristics depth integration |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN208937705U (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110865406A (en) * | 2019-11-28 | 2020-03-06 | 湖南率为控制科技有限公司 | Multi-sensor data synchronous processing system and method based on vehicle-mounted GPS time service system |
| CN114879325A (en) * | 2022-07-11 | 2022-08-09 | 北京精诚恒创科技有限公司 | Built-in type can signal and pass optical cable altogether |
-
2018
- 2018-08-30 CN CN201821413848.7U patent/CN208937705U/en not_active Expired - Fee Related
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110865406A (en) * | 2019-11-28 | 2020-03-06 | 湖南率为控制科技有限公司 | Multi-sensor data synchronous processing system and method based on vehicle-mounted GPS time service system |
| CN114879325A (en) * | 2022-07-11 | 2022-08-09 | 北京精诚恒创科技有限公司 | Built-in type can signal and pass optical cable altogether |
| CN114879325B (en) * | 2022-07-11 | 2022-10-11 | 北京精诚恒创科技有限公司 | Built-in type can signal and pass optical cable altogether |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CA2950791C (en) | Binocular visual navigation system and method based on power robot | |
| Chen et al. | Lidar-histogram for fast road and obstacle detection | |
| CN109444911B (en) | Unmanned ship water surface target detection, identification and positioning method based on monocular camera and laser radar information fusion | |
| CN112396650B (en) | Target ranging system and method based on fusion of image and laser radar | |
| CN102944224B (en) | Work method for automatic environmental perception systemfor remotely piloted vehicle | |
| CN108303096B (en) | Vision-assisted laser positioning system and method | |
| Zhu et al. | Online camera-lidar calibration with sensor semantic information | |
| CN109283538A (en) | A kind of naval target size detection method of view-based access control model and laser sensor data fusion | |
| CN103065323B (en) | Subsection space aligning method based on homography transformational matrix | |
| CN108828621A (en) | Obstacle detection and road surface partitioning algorithm based on three-dimensional laser radar | |
| CN113111887B (en) | Semantic segmentation method and system based on information fusion of camera and laser radar | |
| CN111045000A (en) | Monitoring system and method | |
| CN106774296A (en) | A kind of disorder detection method based on laser radar and ccd video camera information fusion | |
| CN110873879A (en) | Device and method for deep fusion of characteristics of multi-source heterogeneous sensor | |
| CN106598039A (en) | Substation patrol robot obstacle avoidance method based on laser radar | |
| CN112149550A (en) | Automatic driving vehicle 3D target detection method based on multi-sensor fusion | |
| CN111257892A (en) | Obstacle detection method for automatic driving of vehicle | |
| US10860034B1 (en) | Barrier detection | |
| Huang et al. | Tightly-coupled LIDAR and computer vision integration for vehicle detection | |
| CN112379674B (en) | Automatic driving equipment and system | |
| CN108106617A (en) | A kind of unmanned plane automatic obstacle-avoiding method | |
| CN107607953A (en) | A kind of lateral vehicle collision avoidance system and method | |
| CN208937705U (en) | A kind of device of multi-source heterogeneous sensor characteristics depth integration | |
| CN112232139A (en) | Obstacle avoidance method based on combination of Yolo v4 and Tof algorithm | |
| Heckman et al. | Potential negative obstacle detection by occlusion labeling |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20190604 Termination date: 20200830 |