CN110928318A - Binocular vision aided driving system based on FPGA - Google Patents
Binocular vision aided driving system based on FPGA Download PDFInfo
- Publication number
- CN110928318A CN110928318A CN201911421145.8A CN201911421145A CN110928318A CN 110928318 A CN110928318 A CN 110928318A CN 201911421145 A CN201911421145 A CN 201911421145A CN 110928318 A CN110928318 A CN 110928318A
- Authority
- CN
- China
- Prior art keywords
- fpga
- chip
- module
- binocular camera
- binocular
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000013528 artificial neural network Methods 0.000 claims abstract description 11
- 238000013135 deep learning Methods 0.000 claims abstract description 8
- 238000004891 communication Methods 0.000 claims abstract description 6
- 210000003781 tooth socket Anatomy 0.000 claims description 9
- 238000012545 processing Methods 0.000 claims description 7
- 230000007613 environmental effect Effects 0.000 claims description 3
- 238000000605 extraction Methods 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 5
- 238000009434 installation Methods 0.000 description 4
- 238000005259 measurement Methods 0.000 description 3
- 238000001514 detection method Methods 0.000 description 2
- 238000000034 method Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 206010039203 Road traffic accident Diseases 0.000 description 1
- 230000001133 acceleration Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000009897 systematic effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0231—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B19/00—Programme-control systems
- G05B19/02—Programme-control systems electric
- G05B19/04—Programme control other than numerical control, i.e. in sequence controllers or logic controllers
- G05B19/042—Programme control other than numerical control, i.e. in sequence controllers or logic controllers using digital processors
- G05B19/0423—Input/output
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0212—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory
- G05D1/0221—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory involving a learning process
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0231—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
- G05D1/0246—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- Aviation & Aerospace Engineering (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Electromagnetism (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Traffic Control Systems (AREA)
- Studio Devices (AREA)
Abstract
The invention discloses a binocular vision aided driving system based on FPGA, which comprises a shell, wherein an FPGA main chip is arranged in the shell, a deep learning neural network is arranged in the FPGA main chip, the FPGA main chip is connected with a binocular camera, the binocular camera is arranged in the middle of a windshield of a vehicle, the FPGA main chip is further connected with a wireless communication module, a GPS module and a voice module, the binocular camera comprises an FPGA chip, and two ends of the FPGA chip are connected with two cameras. Based on binocular camera, can improve the precision of range finding greatly, binocular camera structure is simple, the angle of regulation camera that can be convenient.
Description
Technical Field
The invention relates to the technical field of fatigue driving detection, in particular to a binocular vision aided driving system based on an FPGA (field programmable gate array).
Background
With the popularity of automobile driving, more and more people choose to drive automobiles. The advanced driving auxiliary system senses the surrounding environment at any time in the driving process of the automobile by using various sensors installed on the automobile, collects data, identifies, detects and tracks static and dynamic objects, and performs systematic operation and analysis by combining with map data of a navigator, so that a driver can be made to perceive possible dangers in advance, the comfort and the safety of automobile driving are effectively improved, and the occurrence of traffic accidents is reduced. The advanced driving assistance system on the market at present has the disadvantages of complex structure, inconvenient use and poor performance. Moreover, the monocular camera is mostly installed on the basis of the vehicle body, has the inherent physical defect of low distance measurement precision, and has poor recognition capability of pedestrians and special-shaped objects. The current binocular camera has the disadvantages of complex structure, inconvenient installation and high cost.
A driving assistance system of utility model patent with publication number CN 209290277U includes a vehicle body; the binocular camera is arranged at the rear end of the vehicle body; the image processing module is connected with the binocular camera; and the display module is connected with the image processing module. The utility model discloses an acquire the image at automobile body rear through two mesh cameras. The structure of a binocular camera is not disclosed, and the driving assistance system is only used for displaying the images and the actual distance value of the obstacle in the images from the vehicle body on the display module at the same time, so that driving assistance in other aspects cannot be performed.
Disclosure of Invention
In order to solve the technical problems, the invention provides a binocular vision aided driving system based on an FPGA (field programmable gate array). based on a binocular camera, the precision of distance measurement can be greatly improved, the structure of the binocular camera is simple, and the angle of the camera can be conveniently adjusted.
The technical scheme adopted by the invention is as follows:
a binocular vision aided driving system based on FPGA comprises a shell, wherein an FPGA main chip is arranged in the shell, a deep learning neural network is arranged in the FPGA main chip, the FPGA main chip is connected with a binocular camera, the binocular camera is arranged in the middle of a windshield of a vehicle, the FPGA main chip is further connected with a wireless communication module, a GPS module and a voice module, the binocular camera comprises an FPGA chip, and two cameras are connected to two ends of the FPGA chip;
the binocular camera is used for sensing the surrounding environment and generating a real-time depth parallax map for three-dimensional reconstruction of the surrounding environment;
the deep learning neural network is internally provided with a trained multitask cascade convolution neural network, the deep disparity map is subjected to real-time environmental feature extraction through the network, the extracted features comprise edge contour information, disparity map depth information and target color information, and a target is detected and identified through the deep disparity map and an original image.
In an optimal technical scheme, the binocular camera further comprises an arm processing chip connected with the FPGA chip, and the arm processing chip is provided with a GPU module.
In a preferred technical scheme, an RJ45 network port, a URAT serial port module, a TF card acquisition module and a USB debugging module are arranged outside the shell, and the RJ45 network port, the URAT serial port module, the TF card acquisition module and the USB debugging module are connected with the FPGA main chip.
In the preferred technical scheme, the binocular camera includes upper cover, lower cover, and sets up the FPGA chip between upper cover and lower cover, the lower cover both ends are provided with the camera installation cavity, the periphery of the outer end camera installation cavity of lower cover is provided with the recess, the FPGA chip is fixed in the lower cover through adjusting device, adjusting device is including setting up installing support and the fixed bolster at both ends in the lower cover, the installing support sets up the pivot mounting hole, the upper end of installing support is provided with the tooth's socket mounting hole, the tooth's socket mounting hole is used for installing the tooth's socket in the pivot mounting hole, the tooth's socket distributes on rotatory path, one side upper end of fixed bolster is provided with the gag lever post, and the opposite side is provided with the pivot, the outer end of pivot is provided with the barb, the pivot is installed in the pivot mounting hole, barb and tooth's.
Compared with the prior art, the invention has the beneficial effects that:
1. the auxiliary driving system is based on the binocular camera, the accuracy and the reliability of sensing of the surrounding environment are guaranteed, the distance measurement precision can be greatly improved, the detection of pedestrians and obstacles can be effectively realized, and the accuracy is higher.
2. The binocular camera has a simple structure, and the angle of the camera can be conveniently adjusted.
Drawings
The invention is further described with reference to the following figures and examples:
FIG. 1 is a schematic block diagram of a binocular vision aided driving system based on an FPGA;
FIG. 2 is a schematic interface diagram of the system of the present invention;
fig. 3 is a schematic overall structure diagram of a binocular camera;
fig. 4 is an exploded view of a binocular camera.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in further detail with reference to the accompanying drawings in conjunction with the following detailed description. It should be understood that the description is intended to be exemplary only, and is not intended to limit the scope of the present invention. Moreover, in the following description, descriptions of well-known structures and techniques are omitted so as to not unnecessarily obscure the concepts of the present invention.
Examples
As shown in fig. 1, the binocular vision aided driving system based on the FPGA comprises a shell, wherein a main FPGA chip is arranged in the shell, a deep learning neural network is arranged in the main FPGA chip, the main FPGA chip is connected with a binocular camera, the binocular camera is arranged in the middle of a windshield of a vehicle, the main FPGA chip is further connected with a wireless communication module, a GPS module and a voice module, the binocular camera comprises a FPGA chip, and two cameras are connected with two ends of the FPGA chip.
And the binocular camera is used for perceiving the surrounding environment, generating a real-time depth parallax map and reconstructing three-dimensional of the surrounding environment.
The GPS/Beidou positioning module is used for positioning the current place through GPS equipment, acquiring longitude and latitude information of the equipment and displaying position information in a map.
The wireless communication module is a 4G communication module and transmits the driving assisting information to the network platform through a 4G network.
The voice module is a loudspeaker, and the output is a control instruction after the target is deeply learned and recognized.
And the deep learning neural network is used for detecting and identifying the target by the depth parallax map and the original image.
The interface schematic diagram of the binocular vision aided driving system is shown in fig. 2. The system comprises: the system comprises an RJ45 network port connected with an FPGA main chip, a URAT serial port module, a TF card acquisition module and a USB debugging module.
The binocular camera adopts two cameras and an FPGA or an arm processing chip with GPU acceleration for 1 block, and realizes the real-time output of depth information.
The structural schematic diagrams of the binocular camera are shown in fig. 3 and 4, and the length, the width and the height of the camera assembly are 185mm, 43mm and 58mm respectively; wherein, the distance of two cameras is 120 mm. The mounting position is generally the right middle position of the vehicle windshield, and the ground clearance is between 1.5m and 2.4 m.
The binocular camera comprises an upper cover 5, a lower cover 1 and an FPGA chip 3 arranged between the upper cover 5 and the lower cover 1, two cameras 11 are arranged at two ends of the FPGA chip 3, camera mounting cavities 12 are arranged at two ends of the lower cover 1, a groove 13 is arranged at the periphery of the outer end camera mounting cavity 12 of the lower cover 1, the FPGA chip 3 is fixed in the lower cover 1 through an adjusting device, the adjusting device comprises a mounting bracket 10 and a fixing bracket 20 which are arranged at two ends in the lower cover 1, the mounting bracket 10 is provided with a rotating shaft mounting hole 14, a tooth socket mounting hole (not shown in the figure) is arranged at the upper end of the mounting bracket 10, the tooth socket mounting hole is used for mounting a tooth socket 15 in the rotating shaft mounting hole 14, the tooth socket 15 can be fixedly mounted through a screw, the tooth socket 15 is distributed on a rotating path, a limiting rod 21 is arranged at the upper end of one side of the fixing bracket 20, the limiting, the outer end of the rotating shaft 22 is provided with a barb, the rotating shaft 22 is arranged in the rotating shaft mounting hole 14, and the barb is matched with the tooth groove. The angle of regulation camera that this regulation structure can be convenient, the periphery of camera installation cavity is provided with the recess for the angle that the camera was adjusted is bigger.
The deep learning neural network selects a trained multitask cascade convolution neural network, and carries out real-time environmental feature extraction on the depth parallax map through the network, wherein the extracted features mainly comprise edge contour information, parallax map depth information and target color information, and pedestrians and obstacles are identified.
It is to be understood that the above-described embodiments of the present invention are merely illustrative of or explaining the principles of the invention and are not to be construed as limiting the invention. Therefore, any modification, equivalent replacement, improvement and the like made without departing from the spirit and scope of the present invention should be included in the protection scope of the present invention. Further, it is intended that the appended claims cover all such variations and modifications as fall within the scope and boundaries of the appended claims or the equivalents of such scope and boundaries.
Claims (4)
1. The binocular vision aided driving system based on the FPGA is characterized by comprising a shell, wherein an FPGA main chip is arranged in the shell, a deep learning neural network is arranged in the FPGA main chip, the FPGA main chip is connected with a binocular camera, the binocular camera is arranged in the middle of a windshield of a vehicle, the FPGA main chip is further connected with a wireless communication module, a GPS module and a voice module, the binocular camera comprises an FPGA chip, and two ends of the FPGA chip are connected with two cameras;
the binocular camera is used for sensing the surrounding environment and generating a real-time depth parallax map for three-dimensional reconstruction of the surrounding environment;
the deep learning neural network is internally provided with a trained multitask cascade convolution neural network, the deep disparity map is subjected to real-time environmental feature extraction through the network, the extracted features comprise edge contour information, disparity map depth information and target color information, and a target is detected and identified through the deep disparity map and an original image.
2. The binocular vision aided driving system based on the FPGA of claim 1, wherein the binocular camera further comprises an arm processing chip connected with the FPGA chip, the arm processing chip being provided with a GPU module.
3. The binocular vision aided driving system based on the FPGA according to claim 1, wherein an RJ45 net port, a URAT serial port module, a TF card acquisition module and a USB debugging module are arranged outside the shell, and the RJ45 net port, the URAT serial port module, the TF card acquisition module and the USB debugging module are connected with an FPGA main chip.
4. The binocular vision aided driving system based on the FPGA according to claim 1, wherein the binocular camera comprises an upper cover, a lower cover and an FPGA chip arranged between the upper cover and the lower cover, camera mounting cavities are arranged at two ends of the lower cover, a groove is arranged at the periphery of the outer end camera mounting cavity of the lower cover, the FPGA chip is fixed in the lower cover through an adjusting device, the adjusting device comprises a mounting bracket and a fixing bracket which are arranged at two ends in the lower cover, the mounting bracket is provided with a rotating shaft mounting hole, a tooth socket mounting hole is arranged at the upper end of the mounting bracket, the tooth socket mounting hole is used for mounting a tooth socket in a rotating shaft mounting hole, the tooth socket is distributed on a rotating path, a limiting rod is arranged at the upper end of one side of the fixing bracket, a rotating shaft is arranged at the other side of the fixing bracket, a barb is arranged at the outer end of the rotating shaft, and, the barb is matched with the tooth groove.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911421145.8A CN110928318A (en) | 2019-12-31 | 2019-12-31 | Binocular vision aided driving system based on FPGA |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911421145.8A CN110928318A (en) | 2019-12-31 | 2019-12-31 | Binocular vision aided driving system based on FPGA |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110928318A true CN110928318A (en) | 2020-03-27 |
Family
ID=69854598
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911421145.8A Pending CN110928318A (en) | 2019-12-31 | 2019-12-31 | Binocular vision aided driving system based on FPGA |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110928318A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111762155A (en) * | 2020-06-09 | 2020-10-13 | 安徽奇点智能新能源汽车有限公司 | Vehicle distance measuring system and method |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20130015743A (en) * | 2011-08-04 | 2013-02-14 | 엘지전자 주식회사 | Vertical angle-adjustable camera device |
| WO2013135052A1 (en) * | 2012-03-15 | 2013-09-19 | 哈尔滨工业大学 | Z-axis lifting mechanism capable of balancing stressed state |
| CN104700385A (en) * | 2013-12-06 | 2015-06-10 | 广西大学 | Binocular vision positioning device based on FPGA |
| CN106767716A (en) * | 2016-12-13 | 2017-05-31 | 云南电网有限责任公司电力科学研究院 | High-tension bus-bar range-measurement system and method based on FPGA hardware and binocular vision |
| CN108556739A (en) * | 2018-03-30 | 2018-09-21 | 东南大学 | Vehicle early warning device based on binocular full-view stereo vision |
| CN108594851A (en) * | 2015-10-22 | 2018-09-28 | 飞智控（天津）科技有限公司 | A kind of autonomous obstacle detection system of unmanned plane based on binocular vision, method and unmanned plane |
| CN108860045A (en) * | 2018-06-28 | 2018-11-23 | 深圳奥尼电子股份有限公司 | Driving assistance method, drive assistance device and storage medium |
| CN208164914U (en) * | 2018-04-25 | 2018-11-30 | 深圳市商汤科技有限公司 | CCD camera assembly and driving assistance system |
| CN208855551U (en) * | 2018-08-23 | 2019-05-14 | 上海势航网络科技有限公司 | A kind of binocular ADAS detector and system |
| CN109747530A (en) * | 2017-11-02 | 2019-05-14 | 郭宇铮 | A kind of dual camera and millimeter wave merge automobile sensory perceptual system |
-
2019
- 2019-12-31 CN CN201911421145.8A patent/CN110928318A/en active Pending
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20130015743A (en) * | 2011-08-04 | 2013-02-14 | 엘지전자 주식회사 | Vertical angle-adjustable camera device |
| WO2013135052A1 (en) * | 2012-03-15 | 2013-09-19 | 哈尔滨工业大学 | Z-axis lifting mechanism capable of balancing stressed state |
| CN104700385A (en) * | 2013-12-06 | 2015-06-10 | 广西大学 | Binocular vision positioning device based on FPGA |
| CN108594851A (en) * | 2015-10-22 | 2018-09-28 | 飞智控（天津）科技有限公司 | A kind of autonomous obstacle detection system of unmanned plane based on binocular vision, method and unmanned plane |
| CN106767716A (en) * | 2016-12-13 | 2017-05-31 | 云南电网有限责任公司电力科学研究院 | High-tension bus-bar range-measurement system and method based on FPGA hardware and binocular vision |
| CN109747530A (en) * | 2017-11-02 | 2019-05-14 | 郭宇铮 | A kind of dual camera and millimeter wave merge automobile sensory perceptual system |
| CN108556739A (en) * | 2018-03-30 | 2018-09-21 | 东南大学 | Vehicle early warning device based on binocular full-view stereo vision |
| CN208164914U (en) * | 2018-04-25 | 2018-11-30 | 深圳市商汤科技有限公司 | CCD camera assembly and driving assistance system |
| CN108860045A (en) * | 2018-06-28 | 2018-11-23 | 深圳奥尼电子股份有限公司 | Driving assistance method, drive assistance device and storage medium |
| CN208855551U (en) * | 2018-08-23 | 2019-05-14 | 上海势航网络科技有限公司 | A kind of binocular ADAS detector and system |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111762155A (en) * | 2020-06-09 | 2020-10-13 | 安徽奇点智能新能源汽车有限公司 | Vehicle distance measuring system and method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109920246B (en) | Collaborative local path planning method based on V2X communication and binocular vision | |
| CN102390370B (en) | Stereoscopic vision based emergency treatment device and method for running vehicles | |
| CN108139211B (en) | Apparatus and method for measurement and program | |
| JP2019045892A (en) | Information processing apparatus, information processing method, program and movable body | |
| JP2020085886A (en) | Vehicle, vehicle positioning system, and method for positioning vehicle | |
| EP3107069A1 (en) | Object detection apparatus, object detection method, and mobile robot | |
| CN110850413A (en) | Method and system for detecting front obstacle of automobile | |
| CN103407407A (en) | Automobile safety distance warning device and method based on binocular stereo vision | |
| CN110826369A (en) | Driver attention detection method and system during driving | |
| CN110126824A (en) | A kind of commercial vehicle AEBS system of integrated binocular camera and millimetre-wave radar | |
| CN111221342A (en) | Environment sensing system for automatic driving automobile | |
| CN104802710A (en) | Intelligent automobile parking assisting system and assisting method | |
| CN106651873B (en) | One kind detecting glasses based on RGB-D camera and stereosonic visually impaired people's zebra stripes | |
| US20190147270A1 (en) | Information processing apparatus, driver monitoring system, information processing method and computer-readable storage medium | |
| CN110909705A (en) | Roadside parking space sensing method and system based on vehicle-mounted camera | |
| US20190147269A1 (en) | Information processing apparatus, driver monitoring system, information processing method and computer-readable storage medium | |
| CN109345591B (en) | Vehicle posture detection method and device | |
| CN111351474A (en) | Vehicle moving target detection method, device and system | |
| CN110928318A (en) | Binocular vision aided driving system based on FPGA | |
| CN111301328A (en) | 3D vision technology-based panoramic car backing image system for truck and use method | |
| CN211827022U (en) | Binocular vision aided driving system based on FPGA | |
| CN107432793A (en) | Safety-type Two-wheeled wheelchair | |
| CN112384952A (en) | Camera calibration method and device and electronic equipment | |
| CN111508232A (en) | Road safety snapshot all-in-one | |
| CN114049794B (en) | Vehicle-mounted safety auxiliary driving system based on smart phone |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |