CN106926800A - The vehicle-mounted visually-perceptible system of multi-cam adaptation - Google Patents
The vehicle-mounted visually-perceptible system of multi-cam adaptation Download PDFInfo
- Publication number
- CN106926800A CN106926800A CN201710193302.9A CN201710193302A CN106926800A CN 106926800 A CN106926800 A CN 106926800A CN 201710193302 A CN201710193302 A CN 201710193302A CN 106926800 A CN106926800 A CN 106926800A
- Authority
- CN
- China
- Prior art keywords
- camera
- vehicle
- unit
- data
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R16/00—Electric or fluid circuits specially adapted for vehicles and not otherwise provided for; Arrangement of elements of electric or fluid circuits specially adapted for vehicles and not otherwise provided for
- B60R16/02—Electric or fluid circuits specially adapted for vehicles and not otherwise provided for; Arrangement of elements of electric or fluid circuits specially adapted for vehicles and not otherwise provided for electric constitutive elements
- B60R16/023—Electric or fluid circuits specially adapted for vehicles and not otherwise provided for; Arrangement of elements of electric or fluid circuits specially adapted for vehicles and not otherwise provided for electric constitutive elements for transmission of signals between vehicle parts or subsystems
- B60R16/0231—Circuits relating to the driving or the functioning of the vehicle
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Automation & Control Theory (AREA)
- Mechanical Engineering (AREA)
- Image Processing (AREA)
- Traffic Control Systems (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a kind of vehicle-mounted visually-perceptible system of multi-cam adaptation, including vehicle-mounted camera signal transmission unit, DSP processing units, camera types judging unit, vision data memory cell and image information pretreatment unit；The present invention can realize the adaptation including the various camera types including infrared camera, monocular cam, monocular wide-angle camera, binocular solid camera, binocular image splicing camera, many lens image spliced panoramic cameras, and set up a kind of universal vehicle-mounted visual processes scheme, for car steering accessory system provides various compatible vision prescription, time cost and financial cost that various vehicle-mounted visions are carried can be equally reduced.
Description
Technical field
The present invention relates in vehicle-mounted visually-perceptible technical field, more particularly to a kind of vehicle-mounted visual impression of multi-cam adaptation
Know system.
Background technology
In recent years, increasingly mature with car steering ancillary technique, various automobile miscellaneous functions more and more should
With on volume production automobile.Car steering ancillary technique is automobile developed to " intellectuality " from " mechanization " one must through technology
Stage；It can provide safety guarantee for driver's driving behavior, while improving comfortableness, security, the fuel oil of vehicle traveling
Economy.In ancillary technique and unmanned technology is driven, environment sensing is its important core component.Environment sensing
Technology refers to vehicle by the coherent signal of the sensor such as including camera, ultrasonic radar, millimetre-wave radar, laser radar to week
Collarette border is perceived, for the control decision of vehicle provides important evidence.Wherein, compared to other environmental sensors, camera
More abundant environmental information, and cheap, dependable performance can be provided, be considered as often unmanned environment sensing skill
One irreplaceable direction of art.
By a kind of sensor group of camera into context aware systems be collectively referred to as vehicle-mounted vision system.Vehicle-mounted vision system
Function have a lot, including driving recording, anti-collision early warning, ACC, lane detection etc..Existing vehicle-mounted vision system only provides list
One function.If driver expects to realize various drive assistance functions, generally require that many set difference in functionalitys are installed on vehicle
The vehicle-mounted vision system of species, inevitably causes a large amount of wastes of hardware and software financial cost.Multiple types are vehicle-mounted simultaneously regards
The compatibling problem and data redundancy issue of feel system, also the strong influence stability of vehicle-mounted vision system, security and
Economy.
In existing vehicle-mounted visually-perceptible system, camera broad categories, are focused/become including infrared/pure visible image capturing head
Burnt camera, common/wide-angle camera, monocular/binocular camera, plane/stereo camera etc..These cameras respectively have respectively
Emphasize advantage, but have certain similarity again in terms of function treatment.Existing vehicle-mounted vision system is not to multiple species
Camera is adapted to, with suitable for various different types of cameras.
In sum, existing vehicle-mounted visually-perceptible system is in multi-class camera adaptation and multimode function integrated approach
Aspect has defect, it is necessary to improve.
The content of the invention
In view of this, it is an object of the invention to provide a kind of vehicle-mounted visually-perceptible system of multi-cam adaptation, to drive
Sail ancillary technique and unmanned technology and more stable, safer, the more integrated vehicle-mounted vision technique of function is provided.
The vehicle-mounted visually-perceptible system of multi-cam adaptation of the invention, including：
Vehicle-mounted camera signal transmission unit, for the signal conversion of vehicle-mounted hardware sensitive component and signal transmission；
DSP processing units, are connected with vehicle-mounted camera signal transmission unit, for the signal of vehicle-mounted camera data signal
Filtering and A/D conversions；
Camera types judging unit, is connected with camera types judging unit, the hardware for recognizing vehicle-mounted camera
Type and for systemic-function provides camera types information；
Vision data memory cell, is connected with DSP processing units, remembers as driving for storing vehicle-mounted camera data
Record, the data-interface that storage image preprocessing result data is merged as sensor；
Image information pretreatment unit, is connected with camera types judging unit and vision data memory cell, for inciting somebody to action
Camera data are pre-processed according to camera types, image is carried out defogging-optimization processing, Shannon entropy detection, gray processing,
Binaryzation and the image segmentation based on rim detection.
Further, the system also includes graphics processing unit, described image processing unit and image information pretreatment unit
It is connected；Described image processing unit includes：
Road surface estimation unit, for being optimized according to defogging after view data vehicle running surface is estimated, is classified；
Visual odometry unit, for according to defogging optimization processing data/greyscale image data, phase being carried out to vehicle traveling
Pose is resolved；
Box counting algorithm unit, for carrying out visual signature resolving to defogging optimization processing data, greyscale image data,
Described including feature detection, feature, calculate feature and be respectively Haar, HOG, FAST, ORB, BRIEF or LBP, the actual solution of feature
Species is calculated to be specified by camera process demand；
Vision map constructing unit, for setting up visual signature map office for BRIEF features, merges with visual odometry
It is visual environment SLAM, treatment is optimized to vision map, carries out real-time winding detection；
Image identification unit, for carrying out vision to vehicle, pedestrian, traffic mark, lane line according to known-image-features
Identification；
Picture charge pattern unit, for position to carry out spy in different frame picture to vehicle, pedestrian, traffic mark, lane line
Matching is levied, calculated, estimated moment state, the position of vehicle, pedestrian, traffic mark, lane line under world coordinate system；And
Visual signature storehouse storage element, for storing the visual signature data that box counting algorithm unit is calculated, storage
Mode is the feature database storing mode according to 3D structural remodelings under sequential, for vision map constructing unit provides calculating data-interface.
Further, the system also includes System Back-end, and the System Back-end is connected with graphics processing unit；After the system
End includes：
Structured road construction unit, for using road surface estimation unit, vision map constructing unit, image identification unit
Calculation result building structure road model, Optimized model result；
Running environment detection unit, for according to road surface estimation unit, vision map constructing unit, image identification unit,
Picture charge pattern cell data calculation result, detection penetrates traveling environmental change in range L rice from rung；And
System data interaction interface unit be used for for it is man-machine drive altogether, man-machine interaction, wagon control decision-making location data extract,
Wagon control policy setting data are extracted and Data Fusion of Sensor decision-making DBMS is extracted and provides data-interface.
Further, the L is 100-200.
Further, the camera types judging unit includes：
Vehicle-mounted visual signal input module, data-interface is judged for providing camera types, by DSP processing unit processes
Consequential signal, infrared generator signal input vehicle-mounted camera type judging unit；
Camera types judge module, for judging camera types, it includes that infrared/pure visible image capturing head judges mould
Block, camera quantity detection module, camera wide-angle detection module and camera view detection module；And
Camera types message output module, for exporting camera judged result.
Further, the system also includes：
Vehicle-mounted visual information input block, is located between camera types judging unit and image information pretreatment unit,
For transmission camera type information and camera installation site information data；And
Vehicle-mounted visual signal input block, is located between camera types judging unit and image information pretreatment unit,
For transmitting original camera data.
Beneficial effects of the present invention：The vehicle-mounted visually-perceptible system of multi-cam adaptation of the invention, can be adapted to various
Camera types, complete the multiple module function of vehicle-mounted vision；In this process, engineering staff only needs few people to work
Industry and manual intervention, save substantial amounts of manpower, time cost；The present invention can realize including that infrared camera, monocular are taken the photograph
As head, monocular wide-angle camera, binocular solid camera, binocular image splicing camera, the shooting of many lens image spliced panoramics
Head and sets up a kind of universal vehicle-mounted visual processes scheme in the adaptation of interior various camera types, is car steering
Accessory system provides various compatible vision prescription, can equally reduce time cost and economy that various vehicle-mounted visions are carried
Cost.
Brief description of the drawings
The invention will be further described with reference to the accompanying drawings and examples：
Fig. 1 is structured flowchart of the invention；
Fig. 2 is the structured flowchart of the camera types judging unit of invention；
Fig. 3 is the structured flowchart of image information pretreatment unit of the invention and image information pretreatment unit；
Fig. 4 is that single vehicle mounted infrared camera image processes thread flow figure；
Fig. 5 is single vehicle-mounted pure visible image capturing head image procossing thread flow figure；
Fig. 6 is multi-cam panoramic picture differentiation process chart；
Fig. 7 is binocular stereo vision image difference alienation process chart；
Fig. 8 is monocular wide-angle camera image difference alienation process chart.
Specific embodiment
This vehicle-mounted visually-perceptible system includes：Vehicle-mounted vision hardware/software system front end, vehicle-mounted vision camera type are sentenced
Disconnected unit, image pre-processing unit, graphics processing unit, vehicle-mounted vision system rear end.
Wherein, vehicle-mounted vision hardware/software system front end includes vehicle-mounted camera hardware installation arrangement, changes, vehicle-mounted to take the photograph
As the conversion of head sensitive component signal, A/D conversions, Digital Signal Processing DSP, camera infrared emittance working condition；It is vehicle-mounted
Vision camera type judging unit includes that camera infrared detection, camera number judge, camera wide-angle is detected, camera
Detect in the visual field；Image pre-processing unit include the demarcation of camera internal reference number, image defogging, the detection of image Shannon entropy, image segmentation,
Picture signal gray processing, image binaryzation；Graphics processing unit includes road surface estimation, feature detection, visual odometry, visually
Figure, pedestrian's identification, pedestrian's tracking, vehicle identification, car tracing, Lane detection, traffic mark identification；After vehicle-mounted vision system
End include vehicle location, structured road build, driving environment state-detection, vehicle-mounted vision system rear end finally be driver/
Pilotless automobile provides driving behavior security warning, control decision foundation, data fusion visual signature, data, decision-making interface.
The actual functional capability of the vehicle-mounted vision system is to judge camera types；According to camera types, to the phase of camera
OFF signal is processed；And the data that final output can directly be used by vehicle-mounted control decision package, or to driver's row
The behavior of sailing carries out security warning；For information fusion provides data, feature, decision-making interface；Simultaneously, to camera processing procedure
The related important results and camera initial data of middle generation are stored.
By vehicle-mounted visually-perceptible system：
Firstth, vehicle-mounted vision camera species is judged, covering camera types has：1) infrared camera-pure visible ray is taken the photograph
As head；2) monocular vision camera-multi-vision visual camera；3) monocular wide-angle camera-monocular common camera；4) binocular stands
Body camera-binocular image splicing-two monocular common cameras of camera；5) many mesh full-view camera-multiple monoculars are common
Camera.
Secondth, according to vehicle-mounted vision camera type and view data, image preprocessing is completed.Gone including image
Mist-optimization processing, Shannon entropy detection, gray processing, binaryzation, the image segmentation based on rim detection.
3rd, according to vehicle-mounted vision camera type, view data and image preprocessing calculation result, complete at image
Reason.Including road surface estimation, feature detection, visual odometry, vision map, pedestrian's identification, pedestrian's tracking, vehicle identification, vehicle
Tracking, Lane detection, traffic mark identification.
4th, according to vehicle-mounted vision camera type, view data and image procossing calculation result, vehicle-mounted vision is completed
System back-end function is realized, including vehicle location, structured road structure, driving environment state-detection.
5th, this vehicle-mounted vision system is finally for driver/pilotless automobile provides driving behavior security warning, control
Decision-making foundation, data fusion visual signature, data, decision-making interface.
The present invention is at large illustrated referring to explanation accompanying drawing and embodiment：
Fig. 1 is structured flowchart of the invention, and system relevant components unit is listed in detail, including：Vehicle-mounted camera signal is passed
Defeated unit 111, DSP processing units 112, camera types judging unit 113, vision data memory cell 114, image information are pre-
Processing unit 115, road surface estimation unit 116, visual odometry unit 120, box counting algorithm unit 121, vision map structure
Build unit 122, image identification unit 123, picture charge pattern unit 124, visual signature storehouse storage element 125, structured road structure
Build unit 130, running environment detection unit 131, system data interaction interface unit 132.
Wherein, vehicle-mounted camera signal transmission unit 111 is used for the conversion of vehicle-mounted hardware sensitive component signal, signal biography
It is defeated.DSP processing units 112 are changed for completing vehicle-mounted camera Digital Signal Processing including signal filtering, A/D.Camera class
Type judging unit 113 is responsible for the type of hardware of the vehicle-mounted various camera adaptation module systems of vision of identification, is vehicle module
Systemic-function provides camera types information.Vision data memory cell 114 is used to store vehicle-mounted camera data as driving
Record, the data-interface that storage image preprocessing result data is merged as sensor, data storage includes：Original camera number
According to, camera defogging-optimization data, camera gradation data, camera binaryzation data, camera Shannon entropy Monitoring Data.
Image information pretreatment unit 115 be used for camera data are pre-processed according to camera types, image is carried out defogging-
Optimization processing, Shannon entropy detection, gray processing, binaryzation, the image segmentation based on rim detection.Road surface estimation unit 116 is used for
Vehicle running surface is estimated according to the view data after defogging optimization, is classified, major function is included；Coefficient of road adhesion
Estimate, road surface change is estimated, structuring/unstructured road is classified, the classification of road material major class.Visual odometry unit 120
For according to defogging optimization processing data/greyscale image data, relative pose resolving being carried out to vehicle traveling.Box counting algorithm
Unit 121 is used to carry out defogging optimization processing data, greyscale image data visual signature resolving, and major function is examined including feature
Survey, feature is described, calculate feature and be respectively Haar, HOG, FAST, ORB, BRIEF, LBP, the actual species that resolves of feature is by imaging
Head process demand is specified.Vision map constructing unit 122 is used to set up visual signature map office for BRIEF features, with vision
Odometer is fused to visual environment SLAM, and treatment is optimized to vision map, carries out real-time winding detection.Image identification unit
123 are used to carry out visual identity to vehicle, pedestrian, traffic mark, lane line according to known-image-features.Picture charge pattern unit
124 for vehicle, pedestrian, traffic mark, lane line, position to carry out characteristic matching in different frame picture, calculates, estimates car
, moment state, the position of pedestrian, traffic mark, lane line under world coordinate system.Visual signature storehouse storage element 125 is used
In the visual signature data that storage box counting algorithm unit 121 is calculated, storage mode is according to 3D structural remodelings under sequential
Feature database storing mode, is the calculating front end of vision map constructing unit 122, for it provides calculating data-interface.Structuring road
Road construction unit 130 is used to be resolved using road surface estimation unit 116, vision map constructing unit 122, image identification unit 123
As a result building structure road model, Optimized model result.Running environment detection unit 131 is used for according to road surface estimation unit
116th, vision map constructing unit 122, image identification unit 123, the data calculation result of picture charge pattern unit 124, detect from car
Radiation scope L meters (L can be 100-200, preferably 150) interior traveling environmental change, including road surface, vehicle, pedestrian, traffic mark
Knowledge, lane line, barrier state change information；Wherein, from camera view in two meters of car radiation scope block in the range of travel
Environmental change, is estimated according to optimal estimating theory EKF methods.System data interaction interface unit 132 is used to be man-machine common
Drive, man-machine interaction, wagon control decision-making location data extract, wagon control policy setting data extract, Data Fusion of Sensor
Decision-making DBMS extraction etc., there is provided data-interface.
Fig. 2 is the structured flowchart of the camera types judging unit of invention, including：Vehicle-mounted visual signal input module 21,
Camera types judge module 22 and camera types message output module 23.
Wherein, vehicle-mounted visual signal input module 21 refers to the result signal of DSP processing units 112, infrared ray
Device signal input vehicle-mounted camera type judging unit 113, there is provided camera types judge data-interface.
Camera types judge module 22, including：Infrared/pure visible image capturing head judge module 221, the inspection of camera quantity
Survey module, camera wide-angle detection module 223, camera view detection module.Wherein infrared/pure visible image capturing head judges mould
Judge that scheme is in block 221：A) infrared generator signal is detected, night vision/without visible light environment visual image data b) is detected
Signal.Camera quantity detection module, according to camera image data, detects camera number.Camera number includes：Singly take the photograph
As first 222, dual camera 224, multi-cam 226.After being detected as dual camera 224 and the situation of multi-cam 226, to camera
Dual camera visual field detection module 225, multi-cam visual field detection module 227 are carried out, detection mode is image local feature
Compare with, topography's Shannon entropy, partial image pixel matching.The testing result of multi-cam visual field detection module 227 judges
It is no for panoramic mosaic camera, dual camera visual field detection module 225 detect whether to be binocular solid camera.Single camera
222 proceed the output result of camera wide-angle detection module 223 is：1) whether it is wide-angle camera, 2) camera wide-angle number
Value.
Camera types message output module 23, camera judged result is exported to the system：1) whether it is infrared photography
Head, 2) camera number, 3) whether be full-view camera, 4) output of full-view camera image mosaic image, 5) whether be binocular
Three-dimensional camera, 6) whether be monocular cam, 7) camera wide-angle numerical value output.
Fig. 3 is the structured flowchart of image information pretreatment unit of the invention and image information pretreatment unit, such as figure institute
Show, vehicle-mounted visual information input block 311 includes camera types information 311a, camera installation site information 311b.Shooting
The step results of head type information 311a output camera types message output module 23.Camera installation site information 311b bags
Include：The vehicle-mounted installation direction setting of vehicle-mounted camera setting height(from bottom), single camera, the vehicle-mounted installation direction setting of multi-camera system,
The vehicle-mounted comprehensive camera types information 311a of visual information input 311, camera installation site information 311b, judge camera car
Carry position.
Vehicle-mounted visual signal input block 312 transmits original camera data.
Image preprocessing 32 is the actual functional capability module of image information pretreatment unit 115, including：Camera internal reference number mark
Fixed 321, image preprocessing functional module 322.Camera internal reference number demarcates 312 schemes two kinds：1) it is manually entered in camera
Parameter value；2) according to vehicle-mounted visual information input 311, vehicle-mounted visual signal input 312, self-calibration is carried out to camera.Image
Preprocessing function module 322 includes：Image defogging-optimization 322e, image gray processing 322a, image binaryzation 322b, image are fragrant
Agriculture entropy detection 322c, image segmentation 322d.The data storage of image preprocessing functional module 322, and for image processing module is provided
Data-interface.
Vehicle-mounted visual pattern processing function module 33 includes：Estimate 331, characteristic processing 332, visual identity 333, regard in road surface
Feel odometer 334, vision map structuring 335, visual pursuit 336.Characteristic processing 332 includes：Image detection 332a, feature description
332b, calculates feature and is respectively Haar, HOG, FAST, ORB, BRIEF, LBP, and calculation result is visual identity 333, vision mileage
Meter 334, vision map structuring 335, visual pursuit 336 provide resolved data and resolve feature.Visual identity 333 includes：Car
Identification 333a, pedestrian identification 333b, Lane detection 333c, traffic mark identification 333d.Visual pursuit 336 is chased after including vehicle
Track 336a, pedestrian follow the trail of 336b, lane line and follow the trail of 336c.Wherein, road surface estimate 331, characteristic processing 332, visual identity 333,
Visual odometry 334, the view data of visual pursuit 336 be image defogging-optimization 322e data, image gray processing 322a data with
And image binaryzation 322b data, the correlation that 312, vehicle-mounted visual information input 311 is provided is demarcated according to camera internal reference number
Data are processed, and specific handling process is as shown in Fig. 4,5,6,7,8.
Vehicle-mounted visual performance module 34 includes：Structured road builds 341, running environment state-detection 342, vehicle location
343.The vehicle-mounted resolved data of visual pattern processing function module 34 is：Estimate from car radiation scope three-dimensional environment coordinate system, road surface
331 data results, the data result of visual identity 333, the data result of vision map structuring 335, the data knot of visual odometry 334
Really, the data result of visual pursuit 336.
Vehicle-mounted vision module system includes information fusion 351, driving safety warning 352, control with external interface module 35
Decision-making 353.Information fusion 351 provides fused data (vision data memory cell 114 is provided), fusion spy for external data fusion
Levy (characteristic processing 332 is provided), fusion decision-making (vehicle-mounted visual performance module 34).Driving safety warning 352, control decision 353
Driver's driving safety is warned according to the vehicle-mounted calculation result of visual performance module 34 or for unmanned intelligent vehicle
It is controlled.
Fig. 4 show single vehicle mounted infrared camera image treatment thread flow figure, workflow start trigger bar in figure
Part is that vehicle-mounted visual information input 311 judges that vehicle-mounted vision system is infrared camera.Single infrared camera data 41 are car
Carry visual signal and be input into 312 data.
The system carries out image storage 421a, camera internal reference number to infrared picture data and demarcates 422b.Image storage
421a stores infrared camera initial data and binaryzation data.The system simultaneous multi-threading is carried out：View data binaryzation
442e is estimated on 431b, feature calculation 441, Shannon entropy detection 432c, image segmentation 433d, road surface.View data binaryzation 431b
Binaryzation data are used for ORB characterizing parts, BRIEF feature calculations part in image storage 421a and feature calculation 441.
Shannon entropy detection 432c, image segmentation 433d are image preprocessing functional module 322, may act on panoramic picture detection, active
Visual performance expands interface.Estimate that 442e is directly used in structured road and builds 472q in road surface.The calculation result of feature calculation 441 is used
In vehicle identification 451f, pedestrian identification 452g, Lane detection 453h, traffic mark identification 454i, visual odometry 455g, regard
Feel map structuring 456k.Vehicle identification 451f, the resolved data of feature calculation 441 and single infrared camera data 41 are used for car
Follow the trail of 461m.Pedestrian's identification 452g, the resolved data of feature calculation 441 and single infrared camera data 41 are chased after for pedestrian
Track 462n.The resolved data of feature calculation 441 and single infrared camera data 41 are used for lane line and follow the trail of 463p.Vision mileage
Meter 455g, vision map structuring 456k resolved datas are used for vehicle location 464.Lane detection 453h, traffic mark identification
454i, lane line are followed the trail of 463p, vision map structuring 456k, road surface estimation 442e calculation results and are built for structured road
472q.Vehicle identification 451f, pedestrian's identification 452g, car tracing 461m, pedestrian follow the trail of 462n calculation results for running environment
State-detection 471.
Fig. 5 show single vehicle-mounted pure visible image capturing head image procossing thread flow figure, and workflow starting is touched in figure
Clockwork spring part is that vehicle-mounted visual information input 311 judges that vehicle-mounted vision system is pure visible image capturing head.Single pure visible image capturing
Head data 51 are that vehicle-mounted visual signal is input into 312 data.
The system carries out image storage 521a, camera internal reference number to infrared picture data and demarcates 522.Image storage 521a
Store pure visible image capturing head initial data, gray processing data and binaryzation data.After visual pattern gray processing 531b, by ash
Degree view data carries out image storage 521a, and carries out next step.
The system simultaneous multi-threading is carried out：View data binaryzation 541c, feature calculation 551, Shannon entropy detection 542d, figure
As 552f is estimated on segmentation 543e, road surface.View data binaryzation 541c binaryzations data are used for image storage 521a and feature
ORB characterizing parts, BRIEF feature calculations part in calculating 551.Shannon entropy detection 542d, image segmentation 543e are image
Preprocessing function module 322, may act on panoramic picture detection, active vision functions expanding interface.Estimate that 552f is direct in road surface
582r is built for structured road.The calculation result of feature calculation 551 is used for vehicle identification 561g, pedestrian's identification 562h, track
Line identification 563i, traffic mark identification 564j, visual odometry 565k, vision map structuring 566m.Vehicle identification 561g, feature
Calculating 551 resolved datas and single pure visible image capturing head data 51 is used for car tracing 571n.Pedestrian's identification 562h, feature
Calculating 551 resolved datas and single pure visible image capturing head data 51 is used for pedestrian and follows the trail of 572p.The solution of feature calculation 551 counts
According to this and single pure visible image capturing head data 51 are used for lane line and follow the trail of 573q.Visual odometry 565k, vision map structuring
566m resolved datas are used for vehicle location 574.Lane detection 563i, traffic mark identification 564j, lane line are followed the trail of 573q, are regarded
Feel that map structuring 566m, road surface estimate that 552f calculation results are used for structured road and build 582r.Vehicle identification 561g, Hang Renshi
Other 562h, car tracing 571n, pedestrian follow the trail of 572p calculation results for running environment state-detection 581.
Fig. 6 is multi-cam panoramic picture differentiation process chart.Camera is determined as multi-cam panorama 61, vehicle-mounted
Visual information input 311 judges that vehicle-mounted vision system is panoramic picture.For pure visible image capturing head then by camera 1 to shooting
Head N initial data image, binary image, gray level image, pan feature and structured road calculation result are spliced, complete
621 and structured road structure result splicing 622 are spelled into image.It is then that camera 1 is former to camera N for infrared camera
Beginning data image, binary image, pan feature and structured road calculation result are spliced, complete image spell 621 with
And structured road builds result splicing 622.
Fig. 7 is binocular stereo vision image difference alienation process chart.Camera is determined as binocular stereo vision 71, vehicle-mounted
Visual information input 311 judges that vehicle-mounted vision system is binocular stereo vision.Then, it is special according to left and right camera data computation vision
3 d space coordinate 72 is levied, while completing visual odometry 731, vision map structuring 732, knot according to binocular stereo vision algorithm
Structure road builds 733.
Fig. 8 is monocular wide-angle camera image difference alienation process chart.Camera is determined as monocular wide-angle camera 81,
Vehicle-mounted visual information input 311 judges that vehicle-mounted vision system is monocular wide-angle camera.Wide-angle is carried out to monocular wide-angle camera
Camera distortion correction 82, and export correction camera signals data 83.
Finally illustrate, the above embodiments are merely illustrative of the technical solutions of the present invention and it is unrestricted, although with reference to compared with
Good embodiment has been described in detail to the present invention, it will be understood by those within the art that, can be to skill of the invention
Art scheme is modified or equivalent, and without deviating from the objective and scope of technical solution of the present invention, it all should cover at this
In the middle of the right of invention.
Claims (6)
1. the vehicle-mounted visually-perceptible system that a kind of multi-cam is adapted to, it is characterised in that including：
Vehicle-mounted camera signal transmission unit, for the signal conversion of vehicle-mounted hardware sensitive component and signal transmission；
DSP processing units, are connected with vehicle-mounted camera signal transmission unit, for the signal filtering of vehicle-mounted camera data signal
And A/D conversions；
Camera types judging unit, is connected with camera types judging unit, the type of hardware for recognizing vehicle-mounted camera
And for systemic-function provides camera types information；
Vision data memory cell, is connected with DSP processing units, for storing vehicle-mounted camera data as driving recording, storage
Deposit the data-interface that image preprocessing result data is merged as sensor；And
Image information pretreatment unit, is connected with camera types judging unit and vision data memory cell, for that will image
Head data are pre-processed according to camera types, and defogging-optimization processing, Shannon entropy detection, gray processing, two-value are carried out to image
Change and the image segmentation based on rim detection.
2. the vehicle-mounted visually-perceptible system that multi-cam according to claim 1 is adapted to, it is characterised in that the system is also wrapped
Graphics processing unit is included, described image processing unit is connected with image information pretreatment unit；Described image processing unit includes：
Road surface estimation unit, for being optimized according to defogging after view data vehicle running surface is estimated, is classified；
Visual odometry unit, for according to defogging optimization processing data/greyscale image data, relative position being carried out to vehicle traveling
Appearance is resolved；
Box counting algorithm unit, for carrying out visual signature resolving to defogging optimization processing data, greyscale image data, including
Feature detection, feature description, calculate feature and are respectively Haar, HOG, FAST, ORB, BRIEF or LBP, and actual resolving of feature is planted
Class is specified by camera process demand；
Vision map constructing unit, for setting up visual signature map office for BRIEF features, is fused to regard with visual odometry
Feel environment SLAM, treatment is optimized to vision map, carry out real-time winding detection；
Image identification unit, for carrying out visual identity to vehicle, pedestrian, traffic mark, lane line according to known-image-features；
Picture charge pattern unit, for position to carry out feature in different frame picture to vehicle, pedestrian, traffic mark, lane line
Match somebody with somebody, calculate, estimate moment state, the position of vehicle, pedestrian, traffic mark, lane line under world coordinate system；And
Visual signature storehouse storage element, for storing the visual signature data that box counting algorithm unit is calculated, storage mode
It is the feature database storing mode according to 3D structural remodelings under sequential, for vision map constructing unit provides calculating data-interface.
3. the vehicle-mounted visually-perceptible system that multi-cam according to claim 2 is adapted to, it is characterised in that the system is also wrapped
System Back-end is included, the System Back-end is connected with graphics processing unit；The System Back-end includes：
Structured road construction unit, for being resolved using road surface estimation unit, vision map constructing unit, image identification unit
As a result building structure road model, Optimized model result；
Running environment detection unit, for according to road surface estimation unit, vision map constructing unit, image identification unit, image
Tracing unit data calculation result, detection penetrates traveling environmental change in range L rice from rung；And
System data interaction interface unit be used for for it is man-machine drive altogether, man-machine interaction, wagon control decision-making location data extract, vehicle
Control decision environmental data is extracted and Data Fusion of Sensor decision-making DBMS is extracted and provides data-interface.
4. the vehicle-mounted visually-perceptible system of multi-cam according to claim 3 adaptation, it is characterised in that the L is
100-200。
5. the vehicle-mounted visually-perceptible system that multi-cam according to claim 3 is adapted to, it is characterised in that the camera
Type judging unit includes：
Vehicle-mounted visual signal input module, data-interface is judged for providing camera types, by DSP processing unit processes results
Signal, infrared generator signal input vehicle-mounted camera type judging unit；
Camera types judge module, for judging camera types, it include infrared/pure visible image capturing head judge module,
Camera quantity detection module, camera wide-angle detection module and camera view detection module；And
Camera types message output module, for exporting camera judged result.
6. the vehicle-mounted visually-perceptible system that multi-cam according to claim 5 is adapted to, it is characterised in that the system is also wrapped
Include：
Vehicle-mounted visual information input block, is located between camera types judging unit and image information pretreatment unit, is used for
Transmission camera type information and camera installation site information data；And
Vehicle-mounted visual signal input block, is located between camera types judging unit and image information pretreatment unit, is used for
Transmit original camera data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710193302.9A CN106926800B (en) | 2017-03-28 | 2017-03-28 | The vehicle-mounted visual perception system of multi-cam adaptation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710193302.9A CN106926800B (en) | 2017-03-28 | 2017-03-28 | The vehicle-mounted visual perception system of multi-cam adaptation |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106926800A true CN106926800A (en) | 2017-07-07 |
| CN106926800B CN106926800B (en) | 2019-06-07 |
Family
ID=59426518
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710193302.9A Expired - Fee Related CN106926800B (en) | 2017-03-28 | 2017-03-28 | The vehicle-mounted visual perception system of multi-cam adaptation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106926800B (en) |
Cited By (24)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107600072A (en) * | 2017-08-31 | 2018-01-19 | 上海科世达-华阳汽车电器有限公司 | A kind of acquisition methods and system of the common preference parameter of more passengers |
| CN107613262A (en) * | 2017-09-30 | 2018-01-19 | 驭势科技（北京）有限公司 | A kind of Vision information processing System and method for |
| CN108195378A (en) * | 2017-12-25 | 2018-06-22 | 北京航天晨信科技有限责任公司 | It is a kind of based on the intelligent vision navigation system for looking around camera |
| CN108259764A (en) * | 2018-03-27 | 2018-07-06 | 百度在线网络技术（北京）有限公司 | Video camera, image processing method and device applied to video camera |
| CN108566529A (en) * | 2018-04-08 | 2018-09-21 | 深圳市沃特沃德股份有限公司 | The method and device of the adaptive camera video format of onboard system |
| CN108609001A (en) * | 2018-05-09 | 2018-10-02 | 上海蓥石汽车技术有限公司 | A kind of design method for pedestrian's anticollision of actively braking |
| CN109188932A (en) * | 2018-08-22 | 2019-01-11 | 吉林大学 | A kind of multi-cam assemblage on-orbit test method and system towards intelligent driving |
| CN109522825A (en) * | 2018-10-31 | 2019-03-26 | 蔚来汽车有限公司 | The Performance Test System and its performance test methods of visual perception system |
| CN109752008A (en) * | 2019-03-05 | 2019-05-14 | 长安大学 | Intelligent vehicle multi-mode co-located system, method and intelligent vehicle |
| CN109752724A (en) * | 2018-12-26 | 2019-05-14 | 珠海市众创芯慧科技有限公司 | A kind of image laser integral type navigation positioning system |
| CN109754415A (en) * | 2017-11-02 | 2019-05-14 | 郭宇铮 | A kind of vehicle-mounted panoramic solid sensory perceptual system based on multiple groups binocular vision |
| CN109801339A (en) * | 2018-12-29 | 2019-05-24 | 百度在线网络技术（北京）有限公司 | Image processing method, device and storage medium |
| CN109842782A (en) * | 2017-11-29 | 2019-06-04 | 深圳市航盛电子股份有限公司 | A kind of vehicle auxiliary test methods, test equipment and storage medium |
| CN109857123A (en) * | 2019-03-21 | 2019-06-07 | 郑州大学 | A kind of fusion method of view-based access control model perception and the indoor SLAM map of laser acquisition |
| CN110136199A (en) * | 2018-11-13 | 2019-08-16 | 北京初速度科技有限公司 | A kind of vehicle location based on camera, the method and apparatus for building figure |
| CN111833627A (en) * | 2019-04-13 | 2020-10-27 | 长沙智能驾驶研究院有限公司 | Vehicle visual range expansion method, device and system and computer equipment |
| CN111860050A (en) * | 2019-04-27 | 2020-10-30 | 北京初速度科技有限公司 | Loop detection method and device based on image frame and vehicle-mounted terminal |
| CN111959400A (en) * | 2020-08-31 | 2020-11-20 | 安徽江淮汽车集团股份有限公司 | Vehicle driving assistance control system and method |
| US20210156951A1 (en) * | 2019-11-26 | 2021-05-27 | Sick Ag | Safety system |
| CN113031594A (en) * | 2021-02-26 | 2021-06-25 | 山东交通学院 | Maintenance operation-based adjoint type active safety warning robot, system and method |
| CN113327428A (en) * | 2021-06-21 | 2021-08-31 | 深圳腾达智能科技有限公司 | Method for rapidly acquiring data of vehicles entering and leaving parking lot |
| CN113525234A (en) * | 2021-07-26 | 2021-10-22 | 北京计算机技术及应用研究所 | Auxiliary driving system device |
| CN113581098A (en) * | 2020-04-30 | 2021-11-02 | 丰田自动车株式会社 | Vehicle control system |
| CN111860050B (en) * | 2019-04-27 | 2024-07-02 | 北京初速度科技有限公司 | Loop detection method and device based on image frames and vehicle-mounted terminal |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2014046748A (en) * | 2012-08-30 | 2014-03-17 | Toyota Motor Corp | Vehicle control device |
| CN103988499A (en) * | 2011-09-27 | 2014-08-13 | 爱信精机株式会社 | Vehicle surroundings monitoring device |
| CN205620323U (en) * | 2016-04-18 | 2016-10-05 | 华南理工大学 | Highway road surface analytic system based on image processing |
| CN205632380U (en) * | 2016-03-30 | 2016-10-12 | 郑州宇通客车股份有限公司 | Vehicle is with multi -functional integrated form initiative safety coefficient and vehicle |
| CN106231284A (en) * | 2016-07-14 | 2016-12-14 | 上海玮舟微电子科技有限公司 | The formation method of 3-D view and system |
| CN106467060A (en) * | 2015-08-20 | 2017-03-01 | Lg电子株式会社 | Display device and the vehicle including this display device |
-
2017
- 2017-03-28 CN CN201710193302.9A patent/CN106926800B/en not_active Expired - Fee Related
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103988499A (en) * | 2011-09-27 | 2014-08-13 | 爱信精机株式会社 | Vehicle surroundings monitoring device |
| JP2014046748A (en) * | 2012-08-30 | 2014-03-17 | Toyota Motor Corp | Vehicle control device |
| CN106467060A (en) * | 2015-08-20 | 2017-03-01 | Lg电子株式会社 | Display device and the vehicle including this display device |
| CN205632380U (en) * | 2016-03-30 | 2016-10-12 | 郑州宇通客车股份有限公司 | Vehicle is with multi -functional integrated form initiative safety coefficient and vehicle |
| CN205620323U (en) * | 2016-04-18 | 2016-10-05 | 华南理工大学 | Highway road surface analytic system based on image processing |
| CN106231284A (en) * | 2016-07-14 | 2016-12-14 | 上海玮舟微电子科技有限公司 | The formation method of 3-D view and system |
Cited By (28)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107600072A (en) * | 2017-08-31 | 2018-01-19 | 上海科世达-华阳汽车电器有限公司 | A kind of acquisition methods and system of the common preference parameter of more passengers |
| CN107613262A (en) * | 2017-09-30 | 2018-01-19 | 驭势科技（北京）有限公司 | A kind of Vision information processing System and method for |
| CN109754415A (en) * | 2017-11-02 | 2019-05-14 | 郭宇铮 | A kind of vehicle-mounted panoramic solid sensory perceptual system based on multiple groups binocular vision |
| CN109842782A (en) * | 2017-11-29 | 2019-06-04 | 深圳市航盛电子股份有限公司 | A kind of vehicle auxiliary test methods, test equipment and storage medium |
| CN108195378A (en) * | 2017-12-25 | 2018-06-22 | 北京航天晨信科技有限责任公司 | It is a kind of based on the intelligent vision navigation system for looking around camera |
| CN108259764A (en) * | 2018-03-27 | 2018-07-06 | 百度在线网络技术（北京）有限公司 | Video camera, image processing method and device applied to video camera |
| CN108566529A (en) * | 2018-04-08 | 2018-09-21 | 深圳市沃特沃德股份有限公司 | The method and device of the adaptive camera video format of onboard system |
| CN108609001A (en) * | 2018-05-09 | 2018-10-02 | 上海蓥石汽车技术有限公司 | A kind of design method for pedestrian's anticollision of actively braking |
| CN109188932A (en) * | 2018-08-22 | 2019-01-11 | 吉林大学 | A kind of multi-cam assemblage on-orbit test method and system towards intelligent driving |
| CN109522825A (en) * | 2018-10-31 | 2019-03-26 | 蔚来汽车有限公司 | The Performance Test System and its performance test methods of visual perception system |
| CN110136199A (en) * | 2018-11-13 | 2019-08-16 | 北京初速度科技有限公司 | A kind of vehicle location based on camera, the method and apparatus for building figure |
| CN110136199B (en) * | 2018-11-13 | 2022-09-13 | 北京魔门塔科技有限公司 | Camera-based vehicle positioning and mapping method and device |
| CN109752724A (en) * | 2018-12-26 | 2019-05-14 | 珠海市众创芯慧科技有限公司 | A kind of image laser integral type navigation positioning system |
| CN109801339A (en) * | 2018-12-29 | 2019-05-24 | 百度在线网络技术（北京）有限公司 | Image processing method, device and storage medium |
| CN109752008A (en) * | 2019-03-05 | 2019-05-14 | 长安大学 | Intelligent vehicle multi-mode co-located system, method and intelligent vehicle |
| CN109857123A (en) * | 2019-03-21 | 2019-06-07 | 郑州大学 | A kind of fusion method of view-based access control model perception and the indoor SLAM map of laser acquisition |
| CN111833627B (en) * | 2019-04-13 | 2022-02-08 | 长沙智能驾驶研究院有限公司 | Vehicle visual range expansion method, device and system and computer equipment |
| CN111833627A (en) * | 2019-04-13 | 2020-10-27 | 长沙智能驾驶研究院有限公司 | Vehicle visual range expansion method, device and system and computer equipment |
| CN111860050A (en) * | 2019-04-27 | 2020-10-30 | 北京初速度科技有限公司 | Loop detection method and device based on image frame and vehicle-mounted terminal |
| CN111860050B (en) * | 2019-04-27 | 2024-07-02 | 北京初速度科技有限公司 | Loop detection method and device based on image frames and vehicle-mounted terminal |
| US20210156951A1 (en) * | 2019-11-26 | 2021-05-27 | Sick Ag | Safety system |
| US11892551B2 (en) * | 2019-11-26 | 2024-02-06 | Sick Ag | Safety system |
| CN113581098A (en) * | 2020-04-30 | 2021-11-02 | 丰田自动车株式会社 | Vehicle control system |
| CN113581098B (en) * | 2020-04-30 | 2024-01-02 | 丰田自动车株式会社 | Control system for vehicle |
| CN111959400A (en) * | 2020-08-31 | 2020-11-20 | 安徽江淮汽车集团股份有限公司 | Vehicle driving assistance control system and method |
| CN113031594A (en) * | 2021-02-26 | 2021-06-25 | 山东交通学院 | Maintenance operation-based adjoint type active safety warning robot, system and method |
| CN113327428A (en) * | 2021-06-21 | 2021-08-31 | 深圳腾达智能科技有限公司 | Method for rapidly acquiring data of vehicles entering and leaving parking lot |
| CN113525234A (en) * | 2021-07-26 | 2021-10-22 | 北京计算机技术及应用研究所 | Auxiliary driving system device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106926800B (en) | 2019-06-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106926800B (en) | The vehicle-mounted visual perception system of multi-cam adaptation | |
| CN109435942B (en) | Information fusion-based parking space line and parking space recognition method and device | |
| CN106845547B (en) | A kind of intelligent automobile positioning and road markings identifying system and method based on camera | |
| CN111986506B (en) | Mechanical parking space parking method based on multi-vision system | |
| CN110930323B (en) | Method and device for removing reflection of image | |
| US10552982B2 (en) | Method for automatically establishing extrinsic parameters of a camera of a vehicle | |
| CN107600067A (en) | A kind of autonomous parking system and method based on more vision inertial navigation fusions | |
| CN112349144B (en) | Monocular vision-based vehicle collision early warning method and system | |
| CN105654732A (en) | Road monitoring system and method based on depth image | |
| CN110850413A (en) | Method and system for detecting front obstacle of automobile | |
| JPH06266828A (en) | Outside monitoring device for vehicle | |
| EP3824623A1 (en) | Camera assessment techniques for autonomous vehicles | |
| CN113071518B (en) | Automatic unmanned driving method, minibus, electronic equipment and storage medium | |
| CN110083099B (en) | Automatic driving architecture system meeting automobile function safety standard and working method | |
| WO2022115987A1 (en) | Method and system for automatic driving data collection and closed-loop management | |
| CN115410181A (en) | Double-head decoupling alignment full scene target detection method, system, device and medium | |
| CN117111085A (en) | Automatic driving automobile road cloud fusion sensing method | |
| CN114818819A (en) | Road obstacle detection method based on millimeter wave radar and visual signal | |
| CN113052071B (en) | Method and system for rapidly detecting distraction behavior of driver of hazardous chemical substance transport vehicle | |
| CN112529011A (en) | Target detection method and related device | |
| CN116403185A (en) | All-weather road surface obstacle detection and early warning method based on vision fusion perception | |
| Zhang et al. | Principles of intelligent automobiles | |
| CN115303263A (en) | Automatic parking method and system based on neural network | |
| TW201816745A (en) | Multifunctional intelligent driving auxiliary recording method and system improves the safety of vehicle during driving | |
| CN117553811B (en) | Vehicle-road co-location navigation method and system based on road side camera and vehicle-mounted GNSS/INS |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20190607 Termination date: 20200328 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |