CN218839315U

CN218839315U - On-vehicle image blind area detecting system of many functional safety

Info

Publication number: CN218839315U
Application number: CN202221269107.2U
Authority: CN
Inventors: 李贵林; 何勇; 刘大建; 张正钊; 贾康; 宋启杰; 姜晓东; 丛日杰
Original assignee: Qingdao Jidong Automobile Parts Co ltd
Current assignee: Qingdao Jidong Automobile Parts Co ltd
Priority date: 2022-05-24
Filing date: 2022-05-24
Publication date: 2023-04-11
Anticipated expiration: 2032-05-24

Abstract

The utility model discloses a many functional safety vehicle mounted image blind area detecting system, its characterized in that: the multifunctional safety vehicle-mounted image blind area detection system comprises a 360-degree panoramic all-around image system, a blind area detection system, a fatigue driving detection system, an electronic rearview mirror and a pedestrian identification system. The utility model has the advantages that: this system is through setting up at vehicle camera all around, realize 360 panorama covers to vehicle environment all around, automobile blind area has effectively been avoided, make the driver can conveniently observe vehicle environment all around, guarantee driving safety, can carry out real-time supervision to driver's gesture, realize driver fatigue detection function, avoid driver fatigue, be equipped with the back vision display screen on vehicle cab's both sides A post respectively, be convenient for driver observation car back and both sides condition, through blind area detecting system, can also be to the whole detections of object and the mark that appear in the camera, make clear and definite warning to the driver.

Description

On-vehicle image blind area detecting system of many functional safety

Technical Field

The utility model relates to a vehicle safety assists technical field, specifically indicates a many functional safety vehicle carries image blind area detecting system.

Background

The automobile blind area is a part of area which is located at a normal driver seat position and cannot be directly observed because the sight line of the driver is shielded by a vehicle body. In short, when a driver sits on a driver seat to drive, a place which cannot be observed is called a blind area of the automobile. In general, the blind areas are divided into an in-vehicle blind area and an out-vehicle blind area. Some blind areas in the vehicle are formed by vehicle structures, and some blind areas are caused by people. The blind area outside the automobile is caused by fixed or moving objects and light problems, and the blind areas encountered by different automobile models are different in size. In the process of vehicle running, traffic accidents caused by the existence of blind areas occur occasionally, and great economic property loss and personal injury are caused to people, so that the multifunctional safety vehicle-mounted image blind area detection system is provided for solving the problem of vehicle blind areas, and vehicles can run more safely.

SUMMERY OF THE UTILITY MODEL

The to-be-solved technical problem be with above the technical problem, provide a visual field range does not have blind area, monitoring effect good many functional safety vehicle mounted image blind area detecting system.

In order to solve the technical problem, the utility model provides a technical scheme does: the utility model provides a many functional safety vehicle mounted image blind area detecting system which characterized in that: the multifunctional safety vehicle-mounted image blind area detection system comprises a 360-degree panoramic all-around image system, a blind area detection system, a fatigue driving detection system, an electronic rearview mirror and a pedestrian recognition system, the multifunctional safety vehicle-mounted image blind area detection system comprises a processor, a plurality of cameras and a display screen, the cameras comprise cameras arranged on a front cover of a vehicle, cameras arranged below rearview mirrors on two sides of the vehicle and respectively positioned in front of and behind the side of the rearview mirror, cameras arranged at the tail of the vehicle, cameras arranged on columns A on two sides outside the vehicle cab and a monitoring camera arranged in the cab and close to the column A on one side of the driver, the display screen is including setting up the back vision display screen on the indoor both sides A post of auttombilism and setting up the well accuse screen on car central control platform, the treater passes through the circuit and is connected with camera and display screen respectively, be equipped with GPU and NPU in the treater and implant panorama concatenation algorithm, degree of deep learning algorithm, 360 panorama surround view image system passes through GPU and panorama concatenation algorithm and realizes 360 panorama surround view image system functions, blind area detecting system, fatigue driving detection system all realize that the blind area detects the function and the fatigue driving detection function through NPU and degree of deep learning algorithm.

As an improvement, the processor is provided with LVDS input interfaces of not less than eight channels, wherein the LVDS input interfaces comprise at least four MIPI-CSI2 interfaces, at least two Parallel-CSI interfaces and at least two USB expansion interfaces, and the processor is provided with an RGB interface and an HDMI interface which are LVDS output interfaces.

As the improvement, the homonymy sets up camera on the column A of the auttombilism room outside and the camera formation of image of the side rear of rear-view mirror below according to upper and lower position demonstration on the back vision display screen on the column A in the homonymy auttombilism room, camera on the column A of the auttombilism room outside forms primary mirror visual effect, the camera of the side rear of rear-view mirror below forms wide-angle mirror visual effect.

As an improvement, the monitoring camera is arranged on an instrument desk in front of a main driver seat.

As an improvement, the multifunctional safety vehicle-mounted image blind area detection system is a split system or an integrated system.

Compared with the prior art, the utility model the advantage lie in: this system is through setting up at vehicle camera all around, realize 360 panorama covers to vehicle environment all around, automobile blind area has effectively been avoided, make the driver can conveniently observe vehicle environment all around, guarantee driving safety, through setting up the surveillance camera who leans on driver one side A post, can carry out real-time supervision to the driver gesture, realize driver fatigue detection function, avoid driver fatigue, influence driving safety, be equipped with the back vision display screen on vehicle cab's both sides A post respectively, the image information that makes the camera of both sides rear-view mirror gather shows on the back vision display screen, be convenient for the driver to observe behind the car and the both sides condition, through blind area detecting system, can also be to appearing the whole detection and mark of object in the camera, clearly and definitely warn the driver.

Drawings

Fig. 1 is the utility model relates to a many functional safety vehicle carries image blind area detecting system's system block diagram.

FIG. 2 is the utility model relates to a visual field blind area sketch map of many functional safety vehicle-mounted image blind area detecting system commercial car.

FIG. 3 is the utility model relates to a many functional safety vehicle carries image blind area detecting system panorama concatenation algorithm's work flow chart.

Fig. 4 is the utility model relates to a virtual 3D of many functional safety vehicle-mounted image blind area detecting system encircles video's schematic diagram.

Fig. 5 is the utility model relates to a 360 panorama panoramic image system of many functional safety vehicle-mounted image blind area detecting system mark the schematic diagram of flow.

Fig. 6 is the utility model relates to a many functional safety vehicle carries image blind area detecting system and marks the schematic diagram in place.

Fig. 7 is a schematic diagram of the vehicle-side early warning range of the multifunctional safety vehicle-mounted image blind area detection system.

Fig. 8 is the utility model relates to a many functional safety vehicle mounted image blind area detecting system both sides camera coverage sketch map.

Fig. 9 is a schematic diagram of object detection of the multifunctional safety vehicle-mounted image blind area detection system of the present invention.

Fig. 10 is the utility model relates to a many functional safety vehicle carries image blind area detecting system fatigue driving and detects the sketch map.

Fig. 11 is the utility model relates to a many functional safety vehicle carries image blind area detecting system fatigue driving detecting system algorithm flow chart.

Detailed Description

The present invention will be described in further detail with reference to the accompanying drawings.

With reference to the accompanying drawings 1-2, a multifunctional safe vehicle-mounted image blind area detection system adopts the product form of an integrated system, namely a 360-degree panoramic all-round-view image system, an electronic rearview mirror, a blind area detection system and a fatigue driving detection system are realized in a mode of a single electronic control unit, and a core processor of the electronic control unit adopts NXP I.MX8android ^TM ,

FreeRTOS,QNX ^TM ,Green/>

Dornerworks XEN ^TM The multifunctional safety vehicle-mounted image blind area detection system supports a multi-channel LVDS high-definition video signal input interface, wherein a camera arranged on a front face cover of a vehicle, cameras arranged in front of and behind side rear-view mirrors below the two side rear-view mirrors and a camera arranged at the tail of the vehicle are processed by a GPU and a panorama splicing algorithm to realize a 360-degree panoramic ring-view image, and video information acquired by side rear cameras on the rear-view mirrors on the left side and the right side is processed by an NPU and a deep learning algorithm to realize functions of double-side blind area detection and an electronic rear-view mirror;the reversing monitoring function is realized through a camera at the tail part of the automobile; the fatigue driving detection function is realized through 1 path of cameras arranged on the column A of the cab; and in addition, the 1-channel video input interface is used as a reserved channel and can be used for monitoring other parts of the vehicle body.

The method comprises the steps that a 4-channel LVDS, a Parallel-CSI interface, a 2-LVDS and a USB extended 2-channel LVDS are accessed through an I.MX8 MIPI-CSI2 interface, and 8-channel LVDS input is counted; LVDS output is respectively realized through an RGB interface and an HDMI interface of the I.MX8, and 2-channel LVDS differential display is calculated; the 360-degree panoramic view image function is realized through a GPU built in the I.MX8and a panoramic stitching algorithm, and the frame rate is more than or equal to 40fps; the fatigue driving detection function is realized through an I.MX8 NPU and a deep learning algorithm, and the computational power consumption is less than or equal to 0.4TOPS; the NPU and the deep learning algorithm of the I.MX8 are adopted to realize the function of detecting blind areas at two sides, and the computational power consumption is less than or equal to 0.8TOPS.

The system supports a 2-channel LVDS high-definition video output interface, wherein one path of video is output to a rear-view display screen on the A column, and the other path of video can be uploaded to a rear platform through a driving recorder.

The working process of the panoramic stitching algorithm is as shown in fig. 3, firstly, camera internal reference calibration is carried out to obtain camera internal parameters and distortion parameters, then external reference calibration is carried out to obtain the installation position, the rotation matrix and the translation matrix of the camera, stitching seam calculation is carried out, fusion area calculation is carried out after perspective change, a mapping table is generated, a starting process is entered, the mapping table is read, if reading is successful, a real-time image is obtained, then the mapping table is inquired, image interpolation calculation is carried out after brightness equalization processing is carried out, finally, a ring view is generated, circulation is continuously carried out from the step of obtaining the real-time image, and if reading of the mapping table fails, camera external reference calibration is carried out again.

The calibration process of the 360-degree panoramic imaging system is shown in fig. 5, firstly, vehicle parameters and camera images are obtained, whether the parameters and the images are normal or not is judged, if the parameters and the images are normal, image ROI area positioning and image preprocessing are carried out, feature point extraction is carried out, wrong feature points are removed, whether feature point extraction is successful or not is judged, if the feature point extraction is successful, camera internal parameters are optimized, calibration results are stored and successful marks are returned, if the extraction is failed, calibration exception analysis is carried out, a calibration state and corresponding description marks are fed back, and if the vehicle parameters and the images are abnormal, the calibration exception analysis is directly carried out.

In a 360-degree panoramic all-round video system, video images around a vehicle are collected through cameras arranged at the front, the rear, the left and the right of the vehicle in 4 directions, a panoramic view around the vehicle body is synthesized by utilizing image fusion and splicing technology, and finally the panoramic view is displayed on a central control screen of a central control console to enlarge the visual field of a driver, and by means of the 360-degree panoramic all-round video system, the driver can directly see whether obstacles exist around the vehicle and the relative direction and distance of the obstacles when sitting in the vehicle, so that the vehicle can be controlled to park in a narrow and congested parking lot or pass through a complex road surface, the accidents of scraping, collision, sinking and the like can be effectively prevented, and simultaneously, the panoramic view can also provide support for algorithms such as recognition, detection and tracking in an automatic driving system, a developed video imaging technology can realize a complete 360-degree surrounding view of a vehicle in real time, video images of the surrounding environment of the vehicle are synthesized through an I.MX83D virtual projection/viewpoint conversion technology, an advanced three-dimensional algorithm can be combined with images of each independent camera smoothly to provide a seamless and clear 360-degree view, specifically, the images of the cameras are sent to a video processing LSI and comprise video capturing and 3D functions, then the camera images are synthesized into a single real-time 3D image and projected to a three-dimensional bowl-shaped grid to generate a virtual 3D surrounding video, and the view angle of the surrounding environment of the vehicle can be converted at will, as shown in FIG. 4.

The following table shows the calibration modes and calibration methods adopted in different stages:

the calibration mode adopted for mass production comprises one-key automatic calibration and manual calibration, the calibration site is shown in figure 6, the illumination of a calibration area is 50-150 lux, the illumination is uniform and shadow-free, in order to ensure the illumination is uniform, the calibration area is suggested to be positioned in an isolation space, a calibration template is a matte paint spray painting, the light reflection rate is below 12.5%, the roughness of the calibration template is 0.2 +/-Ra value <0.4, the stability of the calibration template is strong, the temperature change is small, the wear resistance and general stumbling resistance are realized, the template is required to be not shielded in lens installation, wheel clamping grooves are arranged according to the vehicle model, the vehicle is required to be positioned in the symmetric center of the template, the left error and the right error are less than 3cm, the front error and the back error are less than 5cm, the template error is less than 1cm, and the calibration template is clear and visible.

As shown in fig. 7 to 9, the blind area detection system puts a deep learning algorithm on i.mx8 to operate, acquires target objects such as motor vehicles, riding vehicles and pedestrians after being identified by the deep learning algorithm with real-time images acquired by cameras on two sides of a vehicle as data sources, and decides whether to perform early warning according to the relative position relationship between the target objects and the vehicle, and through the deep learning algorithm, objects such as vehicles, pedestrians, electric vehicles and bicycles appearing in the cameras must be detected, and special scenes such as riding, towing, squatting and blocking must also be detected, for the early warning areas, when the vehicle is moving straight, the early warning areas on the left side and the right side of the vehicle are respectively 0.5 to 10 meters from the side to the vehicle, and the early warning areas in front of and behind the vehicle are respectively GCFB and KCLB areas from a rear-view mirror outside the vehicle to 3 meters behind the vehicle, when the vehicle is turning right, the distance between the BSD on the right side is expanded to be KCSB area of 0.5 to 10 meters, and the early warning delay of the target objects appearing in the early warning areas is not more than 120.

As shown in fig. 10, the fatigue driving detection system is based on the physiological image reaction of the driver, and takes images of the driver through the monitoring camera on the a column in the cab, and uses the facial features, eye signals, head motility, etc. of the driver to infer the fatigue state of the driver, and gives an alarm to prompt and take corresponding measures, and the fatigue driving detection system should be able to detect the following states of the driver: a) Fatigue driving; b) The front is not seen for a long time; c) Smoking; d) Receiving and making a call; e) The driver is not in the driving position; f) Both hands are disengaged from the steering wheel; g) The driver does not fasten a safety belt; h) The interference is considered.

As shown in fig. 11, the algorithm scheme of the fatigue driving detection system is image acquisition, image processing analysis and result output, the image acquisition includes image input and digitization, the image processing analysis includes preprocessing the digitized image, then performing feature extraction and image segmentation, and finally performing image recognition. In summary, those skilled in the art should understand that they should not be limited to the embodiments described above, and that they can design the similar structure and embodiments without departing from the spirit of the invention.

Claims

1. The utility model provides a many functional safety vehicle mounted image blind area detecting system which characterized in that: on-vehicle image blind area detecting system of many functional safety includes 360 panorama all around image systems, blind area detecting system, driver fatigue detecting system, electron rear-view mirror and pedestrian identification system, on-vehicle image blind area detecting system of many functional safety includes treater, a plurality of camera and display screen, the camera is including setting up camera, the setting that sets up in the automobile front face guard below the automobile front face guard and be located the camera of rear-view mirror side the place ahead and side rear respectively, set up at the camera of automobile tail portion, set up the camera on the outer both sides A post of auttombilism and set up the surveillance camera by driver one side A post in the driver's cabin, the display screen is including setting up the back vision display screen on the indoor both sides A post of auttombilism and setting up the well accuse screen on the car center console, the treater passes through the circuit and is connected with camera and display screen respectively.

2. The system according to claim 1, wherein the system comprises: the processor is provided with LVDS input interfaces of not less than eight channels, wherein the LVDS input interfaces comprise at least four MIPI-CSI2 interfaces, at least two Parallel-CSI interfaces and at least two USB expansion interfaces, and the processor is provided with an RGB interface and an HDMI interface which are LVDS output interfaces.

3. The system according to claim 1, wherein the system comprises: the homonymy sets up camera on the vapour vehicle cab outside A post shows on the rear view display screen on the A post in the homonymy vapour vehicle cab according to upper and lower position with the camera formation of image at the side rear of rear-view mirror below, the camera on the vapour vehicle cab outside A post forms primary mirror visual effect, the camera at the side rear of rear-view mirror below forms wide-angle mirror visual effect.

4. The system according to claim 1, wherein the system comprises: the monitoring camera is arranged on an instrument desk in front of the main driver seat.

5. The system according to claim 1, wherein the system comprises: the multifunctional safety vehicle-mounted image blind area detection system is a split system or an integrated system.