CN117068099A

CN117068099A - Rain, snow and fog sensing control system, method, equipment and medium based on vehicle image

Info

Publication number: CN117068099A
Application number: CN202311044261.9A
Authority: CN
Inventors: 张易; 王平; 钱威; 邱聪雨; 张弢
Original assignee: Zhejiang Geely Holding Group Co Ltd; Zhejiang Zeekr Intelligent Technology Co Ltd
Current assignee: Zhejiang Geely Holding Group Co Ltd; Zhejiang Zeekr Intelligent Technology Co Ltd
Priority date: 2023-08-17
Filing date: 2023-08-17
Publication date: 2023-11-17

Abstract

The invention relates to the technical field of vehicle control, and discloses a rain, snow and fog sensing control system, a method, equipment and a medium based on vehicle images, wherein the system comprises: the vehicle image acquisition module is used for acquiring a camera image of the target vehicle; the image analysis module is used for: dividing the image area of the camera image to obtain a target image analysis area; performing feature recognition on the camera image by using a preset environment discrimination algorithm to obtain target features; performing feature evaluation according to the target image analysis area and the target features to obtain an external environment evaluation result, and generating a control signal according to the external environment evaluation result; the rain and snow control module is used for adjusting the rain and snow of the target vehicle according to the control signal; and the snow fog control module is used for prompting the snow fog of the target vehicle according to the control signal. The invention can improve the accuracy of detection and control of the vehicle under the condition of rain, snow and fog.

Description

Rain, snow and fog sensing control system, method, equipment and medium based on vehicle image

Technical Field

The invention relates to the technical field of vehicle control, in particular to a rain, snow and fog sensing control system, method, equipment and medium based on vehicle images.

Background

In the driving process, in order to prevent driving safety accidents, the driving safety in special weather environments such as rain, snow, fog and the like is particularly required to be paid attention, and furthermore, the vehicle sensing and the vehicle control in the rain, snow, fog and the like are very important.

In the prior art, the following methods are adopted for vehicle sensing and control under the conditions of rain, snow, fog and the like:

1. the rain and snow fog sensing is performed based on the flow sensor, but the method has fewer gears when the windshield wiper is controlled, and the rain and snow fog sensing control based on sensing detection cannot be accurately realized;

2. the method is based on the capacitive sensor for sensing rain and snow fog, but the sensor electronic element is easy to damage due to the fact that the sensor is exposed on the surface of a vehicle, so that the detection accuracy is not high enough, and the detection range is limited due to the limitation of the size of the capacitance of the sensor;

3. based on the principle of total reflection of light paths, the method is based on the principle that the infrared sensor is used for sensing rain and snow fog, the transmitting tube transmits infrared rays under the condition of no raindrops, total reflection is realized in the windshield and the glass cavity, and all the infrared rays enter the receiving tube; when raindrops exist on the upper surface of the windshield, the total reflection condition of the light beams at the positions of the raindrops is destroyed, a certain amount of emergent light occurs, the light energy received by the receiving tube is reduced, and the different speeds of the windshield are controlled based on an algorithm; however, when the driving environment light is complex, the receiving tube is easily influenced by the environment light to cause false triggering;

Meanwhile, the sensor sensing detection method is adopted to detect parameters such as raindrop quality, raindrop size, refractive index and the like in a certain area, so that sensing of rainfall, snow brightness, haze and the like is realized, the parameters such as quality, volume, conductivity, refractive index and the like in actual detection can be caused to be different due to the complex influence of composition components after the rainwater in a real environment is fused by sulfur and nitride in the environment, and certain difference exists between the detection based on the sensor detection and actual human eye discrimination, so that detection inaccuracy is caused.

In summary, how to realize the accuracy of detecting and controlling the vehicle in real time under the condition of rain, snow and fog becomes a problem to be solved urgently.

Disclosure of Invention

The invention provides a rain and snow fog sensing control system, a method, electronic equipment and a computer readable storage medium based on a vehicle image, and mainly aims to solve the problem that the accuracy of detection and control of a vehicle under the condition of rain and snow fog is poor.

In order to achieve the above object, the present invention provides a rain, snow and fog sensing control system based on a vehicle image, comprising:

the vehicle image acquisition module is used for acquiring a camera image of the target vehicle;

the image analysis module is used for dividing the image area of the camera image to obtain a target image analysis area; performing feature recognition on the camera image by using a preset environment discrimination algorithm to obtain target features; performing feature evaluation according to the target image analysis area and the target features to obtain an external environment evaluation result, and generating a control signal according to the external environment evaluation result;

The rain and snow control module is used for adjusting the rain and snow of the target vehicle according to the control signal;

and the snow fog control module is used for carrying out snow fog prompt on the target vehicle according to the control signal.

Optionally, when the image analysis module performs the function of dividing the image area of the camera image to obtain the target image analysis area, the image analysis module is specifically configured to:

detecting a feasible region of the camera image according to a preset feasible space detection algorithm;

and selecting an reachable space from the feasible region detection result as a target image analysis region.

Optionally, when the rainy day control is performed, the image analysis module is further specifically configured to, after executing the function of dividing the image area of the camera image to obtain the target image analysis area:

detecting a boundary lane of the camera image to obtain a boundary lane vehicle wheel area image;

detecting the vehicle wheels of the vehicle on the self-lane by the camera image to obtain a vehicle wheel area image of the vehicle on the self-lane;

and taking the boundary lane vehicle wheel area image and the self lane vehicle wheel area image as wheel area images.

Optionally, when the rainy day control is performed, the image analysis module is specifically configured to, when executing the function of performing feature recognition on the camera image by using a preset environment discrimination algorithm to obtain the target feature:

processing the camera image by using a preset Gaussian mixture model to obtain a Gaussian image;

performing differential processing according to the camera image and the Gaussian image to obtain a vehicle foreground image;

and extracting raindrop features from the vehicle foreground image to obtain target features.

Optionally, when the image analysis module performs the function of extracting the raindrop feature from the foreground image of the vehicle to obtain the target feature, the image analysis module is specifically configured to:

carrying out morphological top hat operation on the vehicle foreground image by using a preset type of structural elements to obtain a first characteristic image;

performing differential operation according to the vehicle foreground image and the first characteristic image to obtain a second characteristic image;

and performing open-close operation on the second characteristic image by using a preset second class structural element to obtain a target characteristic.

Optionally, after executing the function of performing feature recognition on the camera image by using a preset environment discrimination algorithm, the image analysis module is specifically further configured to:

Extracting raindrop boundaries in the camera images of the continuous frames by using a preset boundary extraction algorithm;

and carrying out light-dark difference judgment according to the raindrop boundary, and updating the target characteristic according to the result of the light-dark difference judgment to calibrate.

Optionally, when executing the function of performing feature evaluation according to the target image analysis area and the target feature to obtain an external environment evaluation result, the image analysis module is specifically configured to:

according to the target image analysis area, rainwater environment analysis is carried out, and a first rainfall is determined according to the result of the rainwater environment analysis;

determining a second rainfall according to the duty ratio of the target feature in the vehicle foreground image;

and determining a target rainfall according to the first rainfall and the second rainfall, and taking the target rainfall as an external environment evaluation result.

Optionally, when the snow control is performed, the image analysis module is specifically configured to, when executing the function of performing feature evaluation according to the target image analysis area and the target feature to obtain an external environment evaluation result:

performing black-white duty ratio calculation on the target image analysis area to obtain a first snow amount;

Determining a second amount of snow based on a duty cycle of the target feature in the vehicle foreground image;

and determining a target snow amount according to the first snow amount and the second snow amount, and taking the target snow amount as an external environment evaluation result.

Optionally, when the foggy day control is performed, the image analysis module is specifically configured to, when executing the function of performing feature evaluation according to the target image analysis area and the target feature to obtain an external environment evaluation result:

determining a furthest identifiable distance from a marker in the target image analysis area;

performing target edge sharpness analysis on the target features, performing edge recognition and extraction according to the target edge sharpness analysis result, and generating an environmental target boundary index according to the edge extraction result;

performing contrast identification according to the target features and a vehicle body image in the camera image to obtain a target contrast index;

and comprehensively evaluating according to the farthest identifiable distance, the environment target boundary index and the target contrast index to obtain an external environment evaluation result.

In order to solve the above problems, the present invention also provides a rain and snow fog sensing control method based on a vehicle image, the method comprising:

Collecting a camera image of a target vehicle;

dividing the image area of the camera image to obtain a target image analysis area;

performing feature recognition on the camera image by using a preset environment discrimination algorithm to obtain target features;

performing feature evaluation according to the target image analysis area and the target features to obtain an external environment evaluation result, and generating a control signal according to the external environment evaluation result;

and according to the control signal, performing rain and snow regulation and/or snow fog prompt on the target vehicle.

In order to solve the above-mentioned problems, the present invention also provides an electronic apparatus including:

at least one processor; the method comprises the steps of,

a memory communicatively coupled to the at least one processor; wherein,

the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the functions in the vehicle image-based rain and snow fog sensing control system described above.

In order to solve the above-described problems, the present invention also provides a computer-readable storage medium having stored therein at least one computer program that is executed by a processor in an electronic device to realize the functions in the above-described vehicle image-based rain and snow fog sensing control system.

According to the invention, the acquisition of rain, snow and fog information is realized through the cameras around the vehicle; the hardware configuration of a vehicle sensor subsystem is avoided, and the configuration cost of the whole vehicle is reduced; the camera images are subjected to area division and target identification based on an environment discrimination algorithm, so that feature evaluation is realized, an external environment evaluation result is close to the environmental influence of the sense of real human eyes, the evaluation result is more real and accurate, and the judgment of rain, snow and fog is performed through a software algorithm, so that direct contact with external environment factors (such as rainwater, snow and the like) is avoided, the problem of inaccurate judgment caused by raindrop impurities, size differences and change of emissivity is reduced, and the problem of inaccurate detection caused by damage to a sensor by the external environment factors in a traditional detection mode is reduced; the rain and snow control module and the snow fog control module are controlled by the control signals, so that the self-adaptive adjustment of the vehicle on rainy days, snowy days and foggy days based on trigger conditions with different characteristics is realized. Therefore, the rain and snow fog sensing control system, the method, the electronic equipment and the computer readable storage medium based on the vehicle image are mainly used for solving the problem that the accuracy of detection and control of the vehicle under the condition of rain and snow fog is poor.

Drawings

Fig. 1 is a functional block diagram of a vehicle image-based rain and snow fog sensing control system according to an embodiment of the present invention;

fig. 2 is a schematic flow chart of feature recognition of a camera image by using a preset environment discrimination algorithm by an image analysis module according to a first embodiment of the present invention;

fig. 3 is a schematic flow chart of extracting raindrop features from a foreground image of a vehicle by using an image analysis module according to a first embodiment of the present invention;

fig. 4 is a schematic flow chart of a feature evaluation performed by an image analysis module according to a target image analysis area and a target feature according to a second embodiment of the present invention;

fig. 5 is a schematic flow chart of a feature evaluation performed by an image analysis module according to a target image analysis area and a target feature according to a third embodiment of the present invention;

FIG. 6 is a schematic flow chart of a vehicle image-based rain and snow fog sensing control method according to an embodiment of the present invention;

fig. 7 is a schematic structural diagram of an electronic device for implementing the vehicle image-based rain and snow fog sensing control system according to an embodiment of the present invention.

The achievement of the objects, functional features and advantages of the present invention will be further described with reference to the accompanying drawings, in conjunction with the embodiments.

Detailed Description

It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application.

The embodiment of the application provides a rain and snow fog sensing control method based on a vehicle image. The execution subject of the vehicle image-based rain and snow fog sensing control method includes, but is not limited to, at least one of a server, a terminal, and the like, which can be configured to execute the method provided by the embodiment of the application. In other words, the vehicle image-based rain and snow fog sensing control method may be performed by software or hardware installed at a terminal device or a server device. The service end includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, and the like. The server may be an independent server, or may be a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and basic cloud computing services such as big data and artificial intelligence platforms.

The method utilizes a software image processing algorithm to divide areas of images acquired by cameras around a vehicle and simultaneously perform feature analysis to identify water drop coverage areas/image definition in an effective space area; under the condition that the external part of the default vehicle receives rain and is in the same probability, judging the covering condition of the rainwater around the vehicle by using the same judging mode as that of human eyes through a regional sampling scheme, and controlling the rain and the snow according to the covering condition; meanwhile, based on image definition judgment, the state evaluation of the rain, snow and fog vehicles is realized, and meanwhile, the whole vehicle functions corresponding to the vehicles are called to prompt and control.

According to the invention, the collection of rain and snow fog information is realized on a hardware system through sharing cameras around a vehicle; the hardware configuration of a vehicle sensor subsystem is avoided, and the configuration cost of the whole vehicle is reduced; meanwhile, the hardware performance can be converted into software capability, feasibility is established for platform type scheme conversion among different vehicle types, and after the vehicle is off line, the configuration of the rainfall sensing function can be rapidly realized by calibrating camera image software; the invention not only acts on the front-view camera, but also can identify specific areas by the aid of the vehicle periphery camera for the vehicle door and the tail door glass, and different characteristic triggering conditions are formed by means of a rain, snow and fog identification algorithm.

As shown in fig. 1, a functional block diagram of a vehicle image-based rain and snow fog sensing control system according to an embodiment of the present invention is shown. Depending on the functions implemented, the vehicle image-based rain and snow fog sensing control system 100 may include a vehicle image acquisition module 101, an image analysis module 102, a rain and snow control module 103, and a snow fog control module 104. The module of the present invention may also be referred to as a unit, meaning a series of computer program segments capable of being executed by a processor of a vehicle image-based rain, snow and fog sensing control system and performing a fixed function, which are stored in a memory of the vehicle image-based rain, snow and fog sensing control system.

In the present embodiment, the functions concerning the respective modules/units are as follows:

the vehicle image acquisition module 101 is used for acquiring a camera image of a target vehicle;

the image analysis module 102 is configured to divide an image area of the camera image to obtain a target image analysis area; performing feature recognition on the camera image by using a preset environment discrimination algorithm to obtain target features; performing feature evaluation according to the target image analysis area and the target features to obtain an external environment evaluation result, and generating a control signal according to the external environment evaluation result;

the rain and snow control module 103 is used for performing rain and snow adjustment on the target vehicle according to the control signal;

the snow fog control module 104 is configured to perform snow fog prompting on the target vehicle according to the control signal.

The following description will be made with reference to specific embodiments, respectively, regarding respective constituent parts of a vehicle image-based rain and snow fog sensing control system and specific workflows:

example 1

The vehicle image acquisition module 101 is configured to acquire a camera image of a target vehicle.

In the embodiment of the invention, after the system is triggered, the required information around the vehicle is extracted through the vehicle image acquisition module, for example: the residual quantity of front rain shielding, the water mist degree of a front windshield, the residual quantity of vehicle body rainwater and the like; vehicle image acquisition modules include, but are not limited to: and hardware for recording image information (video stream, image stream) on a vehicle video link such as a vehicle forward-looking camera, a panoramic camera, a test camera, a vehicle recorder (DVR) and the like.

Specifically, a front-view camera or a vehicle recorder (DVR) camera is arranged at a position below a front windshield of a target vehicle; compared with the traditional rainfall sensor which directly faces the vehicle wiper and the direct irradiation of ambient light, the vehicle looking around camera is safer in storage environment conditions, the risks of direct scraping and collision are further reduced, and the risks of damage and damage caused by environmental factors are reduced; the invention adopts the camera hardware to shoot the position, such as a front windshield and a vehicle body, which are flat planes, and can effectively remove the dirt on the surface in the vehicle cleaning scene, thereby avoiding the false triggering of signals caused by dirt residues due to complex structure.

In another optional embodiment of the invention, the vehicle driving environment information and the vehicle body surface information can be shot and extracted through vision acquisition hardware in the vehicle or the vehicle internet of things, and the acquired image data is transmitted to the image analysis module; specifically, the vehicle vision acquisition hardware includes, but is not limited to: visible light cameras, TOF cameras (time-of-flight depth cameras), IR cameras (infrared cameras), etc.; visual acquisition hardware in the vehicle electronics includes, but is not limited to: traffic cameras, security cameras, etc.

In the embodiment of the invention, the camera scheme is adopted to realize the detection of rain, snow and fog and form the control system, and compared with the traditional scheme, the camera device is shared on a hardware system, so that the hardware of a rainfall sensor subsystem is saved, and the function construction cost is reduced on the basis of ensuring the function effectiveness.

The image analysis module 102 is configured to divide an image area of the camera image to obtain a target image analysis area; performing feature recognition on the camera image by using a preset environment discrimination algorithm to obtain target features; and performing feature evaluation according to the target image analysis area and the target features to obtain an external environment evaluation result, and generating a control signal according to the external environment evaluation result.

In the embodiment of the invention, after receiving the camera images acquired on each channel, the image analysis module performs image area distinguishing and feature recognition on the camera images in a specific period, and performs feature comprehensive evaluation according to the image area distinguishing result and the feature recognition result to generate a control signal and transmit the control signal to the rain and snow control module.

In the embodiment of the invention, when the rainfall is identified, the comprehensive evaluation of the rainfall can be realized by combining the feasible region analysis in the camera image, the dynamic water mist analysis of the wheels and the characteristic analysis of the weather-proof drops of the vehicle.

In the embodiment of the present invention, when the image analysis module performs the function of dividing the image area of the camera image to obtain the target image analysis area, the image analysis module is specifically configured to:

In the embodiment of the invention, the feasible space detection algorithm can be a general obstacle detection based on image columns, and the region corresponding to the pixels at the bottom of the image is supposed to be drivable under the driving view angle, and extends and grows to the top of the image until encountering an obstacle, so that a complete reachable space is obtained. For example, stick (Stick pixel algorithm) abstracts an obstacle into a Stick (Stick) on the ground, and divides the image space into an reachable space and an obstacle, where Stick is a representation between a pixel and an object, which can be well balanced in terms of accuracy and efficiency.

In another alternative embodiment of the present invention, the feasible spatial detection algorithm may be a Convolutional Neural Network (CNN), the image region division is converted into the semantic segmentation problem through CNN modeling, the method based on semantic segmentation may be divided into Road edge (Road Boundary) semantic segmentation and Road structure (Road Layout) semantic segmentation according to the target modeling mode, for example, HDMapNet (High-Definition Map) may also predict the lane line while outputting the Road edge.

In the embodiment of the present invention, after executing the function of dividing the image area of the camera image to obtain the target image analysis area, the image analysis module is specifically further configured to:

In the embodiment of the invention, in the rain amount identification process, detection of a vehicle wheel area can be further included, wherein the detection of the vehicle wheel area comprises detection of vehicle wheels of a boundary lane and a self lane; for detecting the vehicle wheels of the boundary lane, a background difference scheme and a contour detection scheme can be adopted, and the image of the vehicle wheel area of the boundary lane is obtained by laterally observing the coordinates of the long axis and the short axis of the vehicle wheels and the center of the vehicle wheels; for the detection of the vehicle wheel characteristics of the self-lane, the coordinates of the wheel positions can be acquired for the bottom protruding positions when the vehicle target is identified for the semantics through the detection result of the obstacle, so that the image of the wheel region of the vehicle wheel of the self-lane is acquired.

In the embodiment of the invention, relative to the actual environment in the driving environment, continuous camera images in a certain time period are required to be acquired, and feature extraction is carried out on the camera images of the front windshield; after the collected original image is differentiated from the image processed by the Gaussian model, a vehicle foreground image (namely, the image of a front windshield and raindrops) is obtained; and extracting the characteristics of the raindrops from the extracted vehicle foreground image, and identifying the subsequent raindrops according to the characteristic extraction result.

Referring to fig. 2, in the embodiment of the present invention, when the image analysis module performs the function of identifying features of the camera image by using a preset environment discrimination algorithm to obtain target features, the image analysis module is specifically configured to:

s21, processing the camera image by using a preset Gaussian mixture model to obtain a Gaussian image;

s22, carrying out differential processing according to the camera image and the Gaussian image to obtain a vehicle foreground image;

s23, extracting raindrop features of the vehicle foreground image to obtain target features.

In the embodiment of the invention, after the camera image is obtained, whether each pixel in the camera image accords with the Gaussian model or not can be traversed, and if so, the weight, the expectation and the variance of the corresponding Gaussian model are updated; if not, updating the weight of the corresponding Gaussian model; after updating the corresponding Gaussian models, arranging all the Gaussian models in a descending order according to the ratio of the weight and the variance of the Gaussian models, judging whether the Gaussian models exist in the Gaussian models matched with the pixels in the camera image or not, if not, judging whether the number of the Gaussian models of the current pixel value is equal to a preset number threshold, and if not, creating the Gaussian models of the pixels; if so, updating the last Gaussian model of the pixel; according to the invention, the Gaussian mixture processing is carried out on the camera image through the Gaussian model, and the Gaussian image is obtained.

In the embodiment of the invention, the original camera image and the Gaussian image are subjected to image difference, so that the vehicle foreground image containing the front windshield and the raindrops can be obtained.

In the embodiment of the invention, impurities (such as worm eggs, bird droppings, dust and the like) of front windshield glass in a vehicle foreground image can be filtered through morphological distinction, so that the prepared raindrop characteristics are obtained.

Referring to fig. 3, in the embodiment of the present invention, when the image analysis module performs the function of extracting the raindrop feature from the foreground image of the vehicle to obtain the target feature, the image analysis module is specifically configured to:

s31, carrying out morphological top hat operation on the vehicle foreground image by utilizing a preset type of structural elements to obtain a first characteristic image;

s32, performing differential operation according to the vehicle foreground image and the first characteristic image to obtain a second characteristic image;

s33, performing opening and closing operation on the second characteristic image by using a preset second class structural element to obtain a target characteristic.

In the embodiment of the invention, the structural elements are small in size and can cover round or rectangular dust, cracks and ova in an imaging image generally, and image operators with different sizes are used; the second-class structural elements are structural element size operators taking the maximum occupied diameter as the corresponding rainfall according to Gaussian distribution values, wherein the second-class structural elements are used for collecting images of the sizes of raindrops on a front windshield based on different rainfall, calculating the diameters of the collected raindrops according to mass centers and maximum enveloping circles, arranging the calculated diameters according to gradients; the morphological top hat operation is the difference between the open operation result of the vehicle foreground image and the first characteristic image, wherein the open operation is the operation of firstly corroding and then expanding the characteristic image.

In the embodiment of the present invention, after executing the function of identifying the features of the camera image by using the preset environment discrimination algorithm, the image analysis module is specifically further configured to:

In the embodiment of the invention, because the water drops falling on the front windshield have different brightness distribution in the X/Y direction of the image under different illumination angles; after image acquisition is carried out from a fixed angle, the brightness distribution of a uniform angle can be maintained for water drops dropping from different positions; for the camera image, a boundary extraction algorithm can be adopted to extract each frame of image for boundary extraction, and for the image after boundary extraction, whether the bright and dark differences exist in the region is utilized to carry out target correction on the raindrop bright and dark features so as to improve the accuracy of target feature generation.

In the embodiment of the present invention, when the image analysis module performs the feature evaluation according to the target image analysis area and the target feature to obtain the external environment evaluation result, the image analysis module is specifically configured to:

In the embodiment of the present invention, the target image analysis area may include an image of a wheel area and an image of a feasible region; the image of the feasible region can be identified by adopting image semantics based on the accumulated water of the road surface, and the judgment is completed according to the continuity of the reflection of the feasible region (road surface) and the trend of the road surface texture, so as to determine the rainfall corresponding to the image of the feasible region; the image of the wheel area can be processed by a Gaussian mixture model based on the dynamic rolling-up peripheral water mist of the wheel to generate the water mist characteristics continuously changing in the area, so as to further determine the rainfall corresponding to the image of the wheel area; the first rainfall may be determined based on the rainfall of the wheel area image corresponding to the image of the feasible region.

In the embodiment of the invention, weight calculation can be performed according to the first rainfall and the second rainfall to obtain the target rainfall; the rainy day identification is realized by combining the road surface rainwater condition, the water mist reaction of the wheels and the rainy drop ratio of the windshield, so that the accuracy of rainfall detection is improved.

In the embodiment of the invention, after the target rainfall is generated, the rainfall signal is generated and output to the windshield wiper control module, the windshield wiper control module is controlled based on the rainfall data contained in the rainfall signal, the residual rain drops on the front windshield of the windshield wiper are regulated to be cleaned at different speeds, the rainfall signal can be updated according to the external environment evaluation results obtained by calculation of the camera images at different moments, and after the display degree of the rainfall signal is weakened, the control signal is input to the windshield wiper control module, so that the frequency of the windshield wiper can be automatically reduced, and the self-adaptive rainfall function is realized.

In the embodiment of the invention, the human eyes judge the size of the rainwater, and the speed of the windscreen wiper is determined by observing the coverage area of water drops on the front windscreen and the influence degree of the water drops on the sight line; the invention adopts the same discrimination mechanism and adopts the judgment of the water drop adhesion ratio on the windshield/rainwater adhesion surface, thereby more meeting the evaluation accuracy in the real scene of the vehicle user.

In the embodiment of the invention, compared with the traditional rainfall (flow, capacitance, piezoelectricity and infrared) sensor, the system judgment inaccuracy caused by raindrop impurities, size difference and change of the emissivity is avoided.

The rain and snow control module 103 is configured to perform rain and snow adjustment on the target vehicle according to the control signal.

In the embodiment of the invention, the rain and snow control module comprises a windshield wiper control module, wherein the control signal is a rainfall signal, and after the rainfall signal is transmitted to the windshield wiper control module, the windshield wiper control module can control the whole vehicle windshield wiper control module according to rainfall data contained in the rainfall signal, and adjust the front-end shield residual rain drops of the windshield wiper at different speeds; in addition, according to the weakening of the rainfall signal, the self-adaptive adjustment of the wiper speed can be controlled.

Example two

In the embodiment of the invention, when the snow amount is identified, the comprehensive evaluation of the snow amount can be realized by combining the feasible region analysis in the camera image and the characteristic analysis of the vehicle windshield snow sheet.

In the embodiment of the invention, the target feature is expressed as a snowflake feature, specifically, the process of feature identification in the rainfall identification process is similar, a vehicle foreground image is firstly generated, and the vehicle foreground image is identified according to morphological structural elements constructed based on snow particles (particles and snowflakes), so as to obtain the snowflake feature.

Referring to fig. 4, in the embodiment of the present invention, when the image analysis module performs the feature evaluation according to the target image analysis area and the target feature to obtain the external environment evaluation result, the image analysis module is specifically configured to:

s41, performing black-and-white duty ratio calculation on the target image analysis area to obtain a first snow amount;

s42, determining a second snow amount according to the duty ratio of the target feature in the vehicle foreground image;

s43, determining a target snow amount according to the first snow amount and the second snow amount, and taking the target snow amount as an external environment evaluation result.

In the embodiment of the invention, aiming at the continuous snow degree identification of a feasible region (road surface), the snow degree is calculated by adopting the ratio of the black region to the white region in the acquired image, and the white region is set as snow and the black region is set as the road surface; the black-white area distinguishing threshold value in the target image analysis area needs to be calibrated in an actual environment, and the first snow amount can be determined according to the black-white area distinguishing threshold value.

In the embodiment of the invention, the weight calculation can be performed according to the first snow amount and the second snow amount to obtain the target snow amount; the snow identification is realized by combining the distribution condition of the road surface snow and the snow piece ratio of the windshield, so that the accuracy of snow detection is improved.

In the embodiment of the invention, after the target snow amount is generated, a snow amount signal is generated and output to the windshield wiper control module, the air conditioner control module and the vehicle prompt module, the vehicle windshield wiper control module is controlled based on the snow amount data contained in the snow amount signal, the front-gear residual snow blades of the windshield wiper are adjusted to clean at different speeds, the snow amount signal can be updated according to the external environment evaluation results obtained by calculation of camera images at different moments, and after the display degree of the snow amount signal is weakened, the control signal is input to the windshield wiper control module, so that the frequency of the windshield wiper can be automatically reduced, and the self-adaptive snow amount function is realized; because of the temperature difference between the inside and the outside of the snowfall vehicle, an air conditioner control signal can be generated and transmitted to a vehicle prompt module to control the blowing of a windshield or the blowing of an air conditioner in the vehicle; because the road obstacle is caused by snowfall, the early warning signal can be generated based on the target snow amount and transmitted to the vehicle prompt module for early warning prompt of bad weather, and the probability of accidents is reduced.

In the embodiment of the invention, the rain and snow control module comprises a windshield wiper control module, wherein the control signal is a snow amount signal, and after the snow amount signal is transmitted to the windshield wiper control module, the windshield wiper control module can control the entire vehicle windshield wiper control module according to the snow amount data contained in the snow amount signal, and adjust the windshield wiper to clean the front-gear residual snow pieces at different speeds; in addition, according to the weakening of the snow quantity signal, the self-adaptive adjustment of the windshield wiper speed can be controlled.

In the embodiment of the invention, the snow fog control module comprises an air conditioner control module and a vehicle prompt module, wherein the air conditioner control module can control the blowing of a windshield based on control signals so as to ensure the driving visual field, control or control the blowing of an air conditioner in a vehicle and keep the vehicle at a proper temperature; the vehicle prompt module can control vehicle light, a display screen, sound and the like based on the control signal so as to realize early warning of severe weather.

Example III

When the snow amount is identified, comprehensive evaluation of haze can be realized by combining feasible region analysis in the camera image, edge analysis of target features and vehicle body contrast.

In the embodiment of the invention, the judgment of the fogging degree in the environment can be realized by distinguishing and marking the feasible region and the target in the environment; environmental targets refer to objects in the driving environment that participate in the driving relationship of the vehicle, for example, vehicles, road signs, lane lines, etc. that are present in the environment; environmental objects can be identified and marked by graphical features (rectangular, circular, crescent, etc.).

In the embodiment of the invention, the target recognition can be carried out on the camera image according to the marker such as the road marking according to the haze judgment algorithm, and the target characteristics (such as the road marking, the lane marking and the like) which accord with the marker in the camera image are determined.

Referring to fig. 5, in the embodiment of the present invention, when the image analysis module performs the feature evaluation according to the target image analysis area and the target feature to obtain the external environment evaluation result, the image analysis module is specifically configured to:

s51, determining the farthest identifiable distance according to the marks in the target image analysis area;

s52, carrying out target edge sharpness analysis on the target features, carrying out edge recognition and extraction according to the target edge sharpness analysis result, and generating an environment target boundary index according to the edge extraction result;

S53, carrying out contrast recognition according to the target features and the vehicle body image in the camera image to obtain a target contrast index;

s54, comprehensively evaluating according to the farthest identifiable distance, the environment target boundary index and the target contrast index to obtain an external environment evaluation result.

In the embodiment of the invention, for marks such as road marks, the distance between the vehicle camera and the target is determined by adopting image triangulation ranging, so that the furthest identifiable road mark distance is obtained. For example, the same mark may be photographed multiple times within a certain period of time, and the distance calculation is completed based on the triangular relationship between the photographing time interval and the vehicle camera and the mark.

In the embodiment of the invention, for the target features in the environment, after identification is completed, the sharpness of the target edge is evaluated; and after evaluating the sharpness of the edge of the target, carrying out edge recognition and extraction by an edge enhancement algorithm, and generating a boundary index according to the boundary extraction result. In an actual application scene, for a target with foggy days, boundary extraction does not completely have a breakpoint.

In the embodiment of the invention, the image after target identification in the image is selected, any position in the target area is compared with the fixed position of the vehicle body in contrast, and if the contrast is lower than a certain characteristic value, recording is carried out; after a certain amount of sampling points are carried out in a target, if the characteristic value with low contrast of the sampling points exceeds a certain value, marking the target, specifically, sampling any point in a 30m range in an image exercisable area, and carrying out contrast calculation with a specific position of a vehicle body to obtain a contrast index in the exercisable area.

In the embodiment of the invention, after the comprehensive evaluation of the haze is determined, the haze signal is generated and output to the vehicle prompt module, and the early warning signal can be generated and transmitted to the vehicle prompt module according to the condition that the road is blocked due to foggy days, so that early warning prompt of severe weather is carried out, and the occurrence probability of unexpected situations is reduced.

In the embodiment of the invention, the snow fog control module comprises a vehicle prompt module, and the vehicle prompt module can control vehicle light, a display screen, sound and the like based on control signals so as to realize early warning of severe weather.

In the embodiment of the invention, when a visual scheme is adopted in an actual calibration link or an actual application environment, the difference of refractive indexes and transmittance of rainwater or liquid is not considered, and the target boundary identification result in the calibration process is adopted, so that the detection accuracy can be effectively improved; for different vehicle model calibration results, the background recognition algorithm can be upgraded through software, and the upgrade can be directly pushed through the vehicle OTA; for the image recognition result, finer rain perception and windscreen wiper frequency control can be realized based on control of signal occupation bandwidth, so that a self-adaptive windscreen wiper function is realized; based on rainfall judgment, automatically associating the vehicle warning lamp function in a heavy rain/extra heavy rain state; the driving convenience and the driving safety are improved; after the defogging function is started, the vehicle can adaptively adjust the vehicle window blowing mode based on the judging result, and defogging of the vehicle can be realized before personnel enter the vehicle, so that the driving convenience is improved.

According to the invention, the acquisition of rain, snow and fog information is realized through the cameras around the vehicle; the hardware configuration of a vehicle sensor subsystem is avoided, and the configuration cost of the whole vehicle is reduced; the camera images are subjected to area division and target identification based on an environment discrimination algorithm, so that feature evaluation is realized, an external environment evaluation result is close to the environmental influence of the sense of real human eyes, the evaluation result is more real and accurate, and the judgment of rain, snow and fog is performed through a software algorithm, so that direct contact with external environment factors (such as rainwater, snow and the like) is avoided, the problem of inaccurate judgment caused by raindrop impurities, size differences and change of emissivity is reduced, and the problem of inaccurate detection caused by damage to a sensor by the external environment factors in a traditional detection mode is reduced; the rain and snow control module and the snow fog control module are controlled by the control signals, so that the self-adaptive adjustment of the vehicle on rainy days, snowy days and foggy days based on trigger conditions with different characteristics is realized. Therefore, the rain and snow fog sensing control system based on the vehicle image provided by the invention is mainly used for solving the problem that the accuracy of detection and control of the vehicle is poor under the condition of rain, snow and fog.

Referring to fig. 6, a flow chart of a rain and snow fog sensing control method based on a vehicle image according to an embodiment of the invention is shown. In this embodiment, the rain and snow fog sensing control method based on the vehicle image includes:

s61, acquiring a camera image of a target vehicle;

s62, dividing an image area of the camera image to obtain a target image analysis area;

s63, carrying out feature recognition on the camera image by using a preset environment discrimination algorithm to obtain target features;

s64, performing feature evaluation according to the target image analysis area and the target features to obtain an external environment evaluation result, and generating a control signal according to the external environment evaluation result;

s65, according to the control signal, performing rain and snow adjustment and/or snow fog prompt on the target vehicle.

Fig. 7 is a schematic structural diagram of an electronic device for implementing a rain, snow and fog sensing control system based on a vehicle image according to an embodiment of the present invention.

The electronic device 700 may comprise a processor 701, a memory 702, a communication bus 703 and a communication interface 704, and may further comprise a computer program stored in the memory 702 and executable on the processor 701, such as a rain and snow mist sensing control program based on a vehicle image.

The processor 701 may be formed by an integrated circuit in some embodiments, for example, a single packaged integrated circuit, or may be formed by a plurality of integrated circuits packaged with the same function or different functions, including one or more central processing units (Central Processing Unit, CPU), a microprocessor, a digital processing chip, a graphics processor, a combination of various control chips, and so on. The processor 701 is a Control Unit (Control Unit) of the electronic device, connects respective components of the entire electronic device using various interfaces and lines, executes or executes programs or modules stored in the memory 702 (for example, executes a vehicle image-based rain and snow fog sensing Control program, etc.), and invokes data stored in the memory 702 to perform various functions of the electronic device and process data.

The memory 702 includes at least one type of readable storage medium including flash memory, a removable hard disk, a multimedia card, a card memory (e.g., SD or DX memory, etc.), magnetic memory, magnetic disk, optical disk, etc. The memory 702 may in some embodiments be an internal storage unit of the electronic device, such as a mobile hard disk of the electronic device. The memory 702 may also be an external storage device of the electronic device in other embodiments, for example, a plug-in mobile hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card) or the like. Further, the memory 702 may also include both internal storage units and external storage devices of the electronic device. The memory 702 may be used not only for storing application software installed in an electronic device and various types of data, such as codes of a rain and snow fog sensing control program based on a vehicle image, but also for temporarily storing data that has been output or is to be output.

The communication bus 703 may be a peripheral component interconnect standard (Peripheral Component Interconnect, PCI) bus, or an extended industry standard architecture (Extended Industry Standard Architecture, EISA) bus, among others. The bus may be classified as an address bus, a data bus, a control bus, etc. The bus is arranged to enable connected communication between the memory 702 and the at least one processor 701 etc.

The communication interface 704 is used for communication between the electronic device and other devices, including network interfaces and user interfaces. Optionally, the network interface may include a wired interface and/or a wireless interface (e.g., WI-FI interface, bluetooth interface, etc.), typically used to establish a communication connection between the electronic device and other electronic devices. The user interface may be a Display (Display), an input unit such as a Keyboard (Keyboard), or alternatively a standard wired interface, a wireless interface. Alternatively, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch, or the like. The display may also be referred to as a display screen or display unit, as appropriate, for displaying information processed in the electronic device and for displaying a visual user interface.

Fig. 7 illustrates only an electronic device having components, and it will be appreciated by those skilled in the art that the configuration illustrated in fig. 7 is not limiting of the electronic device 700 and may include fewer or more components than illustrated, or may combine certain components, or a different arrangement of components.

For example, although not shown, the electronic device may further include a power source (such as a battery) for supplying power to the respective components, and preferably, the power source may be logically connected to the at least one processor 701 through a power management device, so that functions of charge management, discharge management, power consumption management, and the like are implemented through the power management device. The power supply may also include one or more of any of a direct current or alternating current power supply, recharging device, power failure detection circuit, power converter or inverter, power status indicator, etc. The electronic device may further include various sensors, bluetooth modules, wi-Fi modules, etc., which are not described herein.

It should be understood that the embodiments described are for illustrative purposes only and are not limited to this configuration in the scope of the patent application.

The rain and snow fog sensing control program based on the vehicle image stored in the memory 702 of the electronic device 700 is a combination of a plurality of instructions, and when executed in the processor 701, can implement the functions in the rain and snow fog sensing control system based on the vehicle image.

In particular, the specific implementation system of the above instructions by the processor 701 may refer to descriptions of related steps in the corresponding embodiments of the drawings, which are not repeated herein.

Further, the integrated modules/units of the electronic device 700 may be stored in a computer readable storage medium if implemented in the form of software functional units and sold or used as a stand alone product. The computer readable storage medium may be volatile or nonvolatile. For example, the computer readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a U disk, a removable hard disk, a magnetic disk, an optical disk, a computer Memory, a Read-Only Memory (ROM).

The present invention also provides a computer-readable storage medium storing a computer program which, when executed by a processor of an electronic device, can realize the functions in the above-described vehicle image-based rain and snow fog sensing control system.

In the several embodiments provided in the present invention, it should be understood that the disclosed apparatus, device and method may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the modules is merely a logical function division, and there may be other manners of division when actually implemented.

The modules described as separate components may or may not be physically separate, and components shown as modules may or may not be physical units, may be located in one place, or may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional module in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units can be realized in a form of hardware or a form of hardware and a form of software functional modules.

It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof.

The present embodiments are, therefore, to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference signs in the claims shall not be construed as limiting the claim concerned.

Furthermore, it is evident that the word "comprising" does not exclude other elements or steps, and that the singular does not exclude a plurality. A plurality of units or means recited in the system claims can also be implemented by means of software or hardware by means of one unit or means. The terms first, second, etc. are used to denote a name, but not any particular order.

Finally, it should be noted that the above-mentioned embodiments are merely for illustrating the technical solution of the present invention and not for limiting the same, and although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications and equivalents may be made to the technical solution of the present invention without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. A vehicle image-based rain and snow fog sensing control system, the system comprising:

2. The vehicle image-based rain and snow fog sensing control system according to claim 1, wherein the image analysis module is specifically configured to, when executing the function of dividing the image area of the camera image to obtain a target image analysis area:

3. The vehicle image-based rain and snow fog sensing control system according to claim 1, wherein, in the rainy day control, the image analysis module is further specifically configured to, after performing the function of dividing the image area of the camera image to obtain a target image analysis area:

4. The vehicle image-based rain and snow fog sensing control system according to claim 1, wherein, when performing the function of performing feature recognition on the camera image by using a preset environment discrimination algorithm to obtain a target feature, the image analysis module is specifically configured to:

5. The vehicle image-based rain and snow fog sensing control system according to claim 4, wherein the image analysis module is specifically configured to, when executing the function of extracting the raindrop feature from the vehicle foreground image to obtain the target feature:

6. The vehicle image-based rain and snow fog sensing control system according to claim 4, wherein the image analysis module is further configured to, after executing the function of performing feature recognition on the camera image by using a preset environment discrimination algorithm to obtain a target feature:

7. The vehicle image-based rain and snow fog sensing control system according to claim 4, wherein the image analysis module is specifically configured to, when executing the function of performing feature evaluation according to the target image analysis area and the target feature to obtain an external environment evaluation result:

8. The vehicle image-based rain and snow fog sensing control system according to claim 1, wherein, when performing the function of performing feature evaluation according to the target image analysis area and the target feature to obtain an external environment evaluation result, the image analysis module is specifically configured to:

9. The vehicle image-based rain and snow fog sensing control system according to claim 1, wherein when the image analysis module performs the feature evaluation according to the target image analysis area and the target feature to obtain the external environment evaluation result, the image analysis module is specifically configured to:

10. A vehicle image-based rain and snow fog sensing control method, characterized in that the method comprises:

collecting a camera image of a target vehicle;

11. An electronic device, the electronic device comprising:

at least one processor; the method comprises the steps of,

a memory communicatively coupled to the at least one processor; wherein,

the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the functions in the vehicle image-based rain and snow mist sensing control system according to any one of claims 1 to 9.

12. A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the functions in the vehicle image-based rain and snow mist sensing control system according to any one of claims 1 to 9.