WO2021036559A1 - 车窗自动清洁方法及装置 - Google Patents

车窗自动清洁方法及装置 Download PDF

Info

Publication number
WO2021036559A1
WO2021036559A1 PCT/CN2020/102184 CN2020102184W WO2021036559A1 WO 2021036559 A1 WO2021036559 A1 WO 2021036559A1 CN 2020102184 W CN2020102184 W CN 2020102184W WO 2021036559 A1 WO2021036559 A1 WO 2021036559A1
Authority
WO
WIPO (PCT)
Prior art keywords
cleaning tool
pixels
image
preset
cleaning
Prior art date
Application number
PCT/CN2020/102184
Other languages
English (en)
French (fr)
Inventor
张峻豪
王改良
黄为
徐文康
黄晓林
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to EP20859291.5A priority Critical patent/EP4011715A4/en
Publication of WO2021036559A1 publication Critical patent/WO2021036559A1/zh
Priority to US17/678,334 priority patent/US20220176912A1/en

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60S—SERVICING, CLEANING, REPAIRING, SUPPORTING, LIFTING, OR MANOEUVRING OF VEHICLES, NOT OTHERWISE PROVIDED FOR
    • B60S1/00—Cleaning of vehicles
    • B60S1/02—Cleaning windscreens, windows or optical devices
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60S—SERVICING, CLEANING, REPAIRING, SUPPORTING, LIFTING, OR MANOEUVRING OF VEHICLES, NOT OTHERWISE PROVIDED FOR
    • B60S1/00—Cleaning of vehicles
    • B60S1/02—Cleaning windscreens, windows or optical devices
    • B60S1/04—Wipers or the like, e.g. scrapers
    • B60S1/06—Wipers or the like, e.g. scrapers characterised by the drive
    • B60S1/08—Wipers or the like, e.g. scrapers characterised by the drive electrically driven
    • B60S1/0818—Wipers or the like, e.g. scrapers characterised by the drive electrically driven including control systems responsive to external conditions, e.g. by detection of moisture, dirt or the like
    • B60S1/0822—Wipers or the like, e.g. scrapers characterised by the drive electrically driven including control systems responsive to external conditions, e.g. by detection of moisture, dirt or the like characterized by the arrangement or type of detection means
    • B60S1/0833—Optical rain sensor
    • B60S1/0844—Optical rain sensor including a camera
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60H—ARRANGEMENTS OF HEATING, COOLING, VENTILATING OR OTHER AIR-TREATING DEVICES SPECIALLY ADAPTED FOR PASSENGER OR GOODS SPACES OF VEHICLES
    • B60H1/00—Heating, cooling or ventilating [HVAC] devices
    • B60H1/00642—Control systems or circuits; Control members or indication devices for heating, cooling or ventilating devices
    • B60H1/00735—Control systems or circuits characterised by their input, i.e. by the detection, measurement or calculation of particular conditions, e.g. signal treatment, dynamic models
    • B60H1/00785—Control systems or circuits characterised by their input, i.e. by the detection, measurement or calculation of particular conditions, e.g. signal treatment, dynamic models by the detection of humidity or frost
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60S—SERVICING, CLEANING, REPAIRING, SUPPORTING, LIFTING, OR MANOEUVRING OF VEHICLES, NOT OTHERWISE PROVIDED FOR
    • B60S1/00—Cleaning of vehicles
    • B60S1/02—Cleaning windscreens, windows or optical devices
    • B60S1/04—Wipers or the like, e.g. scrapers
    • B60S1/06—Wipers or the like, e.g. scrapers characterised by the drive
    • B60S1/08—Wipers or the like, e.g. scrapers characterised by the drive electrically driven
    • B60S1/0896—Wipers or the like, e.g. scrapers characterised by the drive electrically driven including control systems responsive to a vehicle driving condition, e.g. speed
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60S—SERVICING, CLEANING, REPAIRING, SUPPORTING, LIFTING, OR MANOEUVRING OF VEHICLES, NOT OTHERWISE PROVIDED FOR
    • B60S1/00—Cleaning of vehicles
    • B60S1/02—Cleaning windscreens, windows or optical devices
    • B60S1/46—Cleaning windscreens, windows or optical devices using liquid; Windscreen washers
    • B60S1/48—Liquid supply therefor
    • B60S1/481—Liquid supply therefor the operation of at least part of the liquid supply being controlled by electric means
    • B60S1/485—Liquid supply therefor the operation of at least part of the liquid supply being controlled by electric means including control systems responsive to external conditions, e.g. by detection of moisture, dirt or the like
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/56—Extraction of image or video features relating to colour
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads

Definitions

  • This application relates to the field of communication technology, and in particular to a method and device for automatically cleaning vehicle windows.
  • the present application provides an automatic cleaning method for vehicle windows, so as to realize automatic cleaning of vehicle windows and improve the efficiency of cleaning sundries on the vehicle windows.
  • an embodiment of the present application provides a method for automatically cleaning a vehicle window.
  • the method is applied to a vehicle, and the method includes: acquiring a vehicle window image. Determine the dark channel image corresponding to a single frame of car window image, and determine the presence of the first type of debris on the car window according to the number of pixels in the dark channel image whose gray value exceeds the preset gray threshold and/or the clarity of the dark channel image And/or according to the RGB values of the pixels in the consecutive i frames of the car window image, it is determined that there is a second type of debris on the car window. Control the cleaning tool to remove the first type of debris and/or the second type of debris.
  • the gray value of the pixel in the dark channel image corresponding to a single frame of the window image is used to detect the first type of debris on the window, and/or The RGB values of the pixels in the consecutive i frames of the car window image are used to detect the second type of debris on the car window. If there are first-type debris and/or second-type debris on the window, control the cleaning tool to remove the debris. First of all, cleaning after detection of debris can reduce blind cleaning, improve cleaning efficiency, and save cleaning resources. In addition, through the foregoing process, the embodiments of the present application can realize automatic cleaning of the vehicle windows, reduce manual operations, and improve safety during driving of the vehicle.
  • determining the presence of the second type of debris on the car window specifically includes: first according to the RGB values of the pixels in the consecutive i frames of the car window image , Establish an image background model. Then, according to the RGB values of the pixels in the image background model, the first area is determined, and then the second type of debris is determined in the first area.
  • the RGB value of the pixel in the image background model represents the change in the RGB value of the pixel at the same position in the consecutive i frames of the window image, i is an integer greater than 1, and the first area is the RGB value of the consecutive i frames of the window image. The window area corresponding to the changed pixel.
  • the picture (pixel value) of the position other than the fallen leaves on the window will be changed.
  • the picture (pixel value) where the leaves are present on the window will not change or the degree of change will be small as the vehicle travels.
  • the first area where the second type of debris exists on the window can be determined, which can improve Determine the accuracy of the second type of debris on the window, and determine the accuracy of the first area where the second type of debris is on the window.
  • the method further includes: determining the first area on each of the consecutive j frames of the window image. A second area corresponding to the area, and j second areas are obtained. If the RGB values of the pixels at the same position in the j second regions have not changed, it is determined that there is a second type of debris in the first region.
  • consecutive j frames of window images are located after consecutive i frames of window images, and j is an integer greater than 1.
  • the definition of the dark channel image is the variance of the gray value of the pixels in the dark channel image.
  • the image background model is established according to the RGB values of pixels in consecutive i frames of car window images, which specifically includes: determining the continuous RGB values of pixels at the same position in consecutive i frames of car window images. The mean image of frame i car window images. Then, an image background model is established based on the difference between the RGB values of the pixels in the mean image of the consecutive i frames of the car window image and the RGB values of the pixels in the i-th frame of the car window image.
  • determining the first area according to the RGB values of the pixels in the image background model specifically includes: determining the first area according to the RGB values of the pixels in the image background model and the first preset algorithm.
  • the first preset algorithm is a saliency detection algorithm or a target detection algorithm.
  • the method before determining that the second type of debris exists in the first area, the method further includes: first determining a third area corresponding to the first area in the image background model, and then determining each second area The difference between the RGB value of the middle pixel and the RGB value of the pixel at the same position in the third area is accumulated to obtain an accumulated value. If the accumulated value exceeds the preset threshold, it is determined that the RGB values of the pixels at the same position in the j second regions have changed.
  • controlling the cleaning tool to remove debris includes: controlling the first cleaning tool to be turned on for a preset period of time according to the temperature data inside and outside the vehicle, removing the first type of debris, and/or first according to the driving state and/ Or the first preset weight parameter corresponding to the driver’s state, the driving state and/or the driver’s state, as well as the maximum duration of continuous operation of the second cleaning tool, determine the working time of the second cleaning tool, and then according to the driving state and/or the driver
  • the second preset weight parameter corresponding to the state, the driving state and/or the driver state, and the maximum frequency of the second cleaning tool working determine the working frequency of the second cleaning tool, and finally according to the working time of the second cleaning tool and the second cleaning tool
  • the working frequency controls the second cleaning tool to clean the first area and remove the second type of debris.
  • the working time and working frequency of the second cleaning tool are adaptively adjusted, which can minimize the reduction.
  • the impact of window cleaning on the driver ensures the safety of the vehicle during driving.
  • the second cleaning tool is determined
  • the working time of the cleaning tool specifically includes: according to the driving state, the driver state, the first preset weight parameter corresponding to the driving state, the first preset weight parameter corresponding to the driver state, the maximum continuous working time of the second cleaning tool, and the first preset weight parameter corresponding to the driving state.
  • the preset algorithm determines the working time of the second cleaning tool.
  • the second preset algorithm is t 1 is the working time of the second cleaning tool, t s-max is the maximum continuous working time of the second cleaning tool, e is the base of the natural logarithm, ⁇ 1 and ⁇ 1 are the first corresponding to the driving state and the driver state, respectively.
  • the second cleaning tool is determined according to the driving state and/or the driver state, the second preset weight parameter corresponding to the driving state and/or the driver state, and the maximum frequency of the second cleaning tool.
  • the working frequency of the tool specifically includes: according to the driving state, the driver state, the second preset weight parameter corresponding to the driving state, the second preset weight parameter corresponding to the driver state, the maximum frequency at which the second cleaning tool works, and the third preset Set up an algorithm to determine the working frequency of the second cleaning tool.
  • the method further includes: first, driving according to the driving state and/or the driver state.
  • the third preset weight parameter corresponding to the state and/or the driver state, and the maximum duration of continuous operation of the third cleaning tool determine the working duration of the third cleaning tool.
  • the working interval of the third cleaning tool is determined according to the driving state and/or the driver state, the fourth preset weight parameter corresponding to the driving state and the driver state, and the maximum working interval of the third cleaning tool.
  • the third cleaning tool is controlled to clean the first area according to the working time of the third cleaning tool and the working interval of the third cleaning tool
  • the second cleaning tool is controlled to clean the first area according to the working time of the second cleaning tool and the working frequency of the second cleaning tool. Remove the second type of debris.
  • the working time and working frequency of the second cleaning tool are adaptively adjusted to the smallest extent. Reduce the impact of window cleaning on the driver and ensure the safety of the vehicle during driving. Using a variety of cleaning tools to automatically clean the windows can maximize the cleaning efficiency, thereby further improving the safety of the vehicle during driving.
  • the third preset weight parameter corresponding to the driving state and/or the driver state, the driving state and/or the driver state, and the maximum duration of continuous operation of the third cleaning tool are determined according to the third preset weight parameter.
  • the working time of the cleaning tool specifically includes: according to the driving state, the driver state, the third preset weight parameter corresponding to the driving state, the third preset weight parameter corresponding to the driver state, the maximum continuous working time of the third cleaning tool, and the third preset weight parameter corresponding to the driving state. Four preset algorithms to determine the working time of the third cleaning tool.
  • the fourth preset algorithm is t 2 is the working time of the third cleaning tool, t l-max is the maximum continuous working time of the third cleaning tool, e is the base of the natural logarithm, and ⁇ 3 and ⁇ 3 are the driving state and the driver state, respectively
  • the corresponding third preset weight parameter, ⁇ 3 + ⁇ 3 1, s 1 represents the driving state, and s 2 represents the driver's state.
  • the third cleaning tool is determined to work according to the driving state and/or the driver state, the fourth preset weight parameter corresponding to the driving state and the driver state, and the maximum interval of the third cleaning tool.
  • the interval includes: according to the driving state, the driver state, the fourth preset weight parameter corresponding to the driving state, the fourth preset weight parameter corresponding to the driver state, the maximum working interval of the third cleaning tool, and the fifth preset algorithm , Determine the working interval of the third cleaning tool.
  • the fifth preset algorithm is ⁇ t is the working interval of the third cleaning tool, ⁇ t max is the maximum working interval of the third cleaning tool, e is the base of the natural logarithm, ⁇ 4 and ⁇ 4 are the fourth preset corresponding to the driving state and the driver state respectively
  • the weight parameter, ⁇ 4 + ⁇ 4 1, s 1 represents the driving state, and s 2 represents the driver's state.
  • an embodiment of the present application provides an automatic window cleaning device.
  • the device is applied to a vehicle.
  • the method includes an acquisition unit, a determination unit, and a control unit.
  • the acquiring unit is used to acquire the image of the car window.
  • the determining unit is used to determine the dark channel image corresponding to the single frame of the car window image.
  • the determining unit is further configured to determine the presence of the first type of debris on the car window according to the number of pixels in the dark channel image whose gray value exceeds the preset gray threshold and/or the clarity of the dark channel image; and/or the determining unit , Is also used to determine the presence of the second type of debris on the window according to the RGB values of the pixels in the consecutive i frames of the window image.
  • the control unit is used to control the cleaning tool to remove the first type of debris and/or the second type of debris.
  • the determining unit is specifically configured to first establish an image background model according to the RGB values of pixels in the continuous i-frame car window image. Then, according to the RGB values of the pixels in the image background model, the first area is determined, and then the second type of debris is determined in the first area.
  • the first area is the window area corresponding to the pixels whose RGB value has not changed in consecutive i frames of window image
  • the RGB value of the pixel in the image background model represents the RGB value of the pixel at the same position in consecutive i frames of window image
  • the change of i is an integer greater than 1.
  • the determining unit after determining the first area according to the RGB values of the pixels in the image background model, is also used to determine on each frame of the continuous j frames of window images A second area corresponding to the first area obtains j second areas. If the RGB values of the pixels at the same position in the j second regions have not changed, it is determined that there is a second type of debris in the first region.
  • consecutive j frames of window images are located after consecutive i frames of window images, and j is an integer greater than 1.
  • the determining unit is specifically configured to: if the number of pixels in the dark channel image whose gray value does not exceed the preset gray threshold value is less than or equal to the number of pixels in the dark channel image whose gray value exceeds the preset gray threshold value The number of pixels, and/or the sharpness of the dark channel image is less than the preset sharpness, it is determined that there is a first type of debris on the car window.
  • the definition of the dark channel image is the variance of the gray value of the pixels in the dark channel image.
  • the determining unit is specifically configured to determine the mean value image of the continuous i frames of the car window image according to the mean value of the RGB values of the pixels at the same position in the continuous i frames of the car window image. Then, an image background model is established based on the difference between the RGB values of the pixels in the mean image of the consecutive i frames of the car window image and the RGB values of the pixels in the i-th frame of the car window image.
  • the determining unit is specifically configured to determine the first region according to the RGB values of the pixels in the image background model and the first preset algorithm.
  • the first preset algorithm is a saliency detection algorithm or a target detection algorithm.
  • the determining unit is also used to first determine a third area corresponding to the first area in the image background model, and then determine that the RGB value of each pixel in the second area is the same as that in the third area. The difference of the RGB value of the pixel at the position is accumulated and the accumulated value is obtained. If the accumulated value exceeds the preset threshold, it is determined that the RGB values of the pixels at the same position in the j second regions have changed.
  • control unit is specifically configured to control the first cleaning tool to be turned on for a preset duration according to the temperature data inside and outside the vehicle, remove the first type of debris, and/or first according to the driving state and/or the driver state ,
  • the driving state and/or the first preset weight parameter corresponding to the driver state, and the maximum duration of the second cleaning tool's continuous working time determine the working time of the second cleaning tool, and then according to the driving state and/or driver state, driving state And/or the second preset weight parameter corresponding to the driver’s state, and the maximum frequency of the second cleaning tool, determine the working frequency of the second cleaning tool, and finally control the second cleaning tool according to the working time of the second cleaning tool and the working frequency of the second cleaning tool.
  • the second cleaning tool cleans the first area and removes the second type of debris.
  • control unit is specifically used for the first preset weight parameter corresponding to the driving state and/or the driver state, the driving state and/or the driver state, and the continuous working of the second cleaning tool
  • the maximum duration is to determine the working duration of the second cleaning tool, which specifically includes: according to the driving state, the driver state, the first preset weight parameter corresponding to the driving state, the first preset weight parameter corresponding to the driver state, and the second cleaning tool continuous
  • the maximum working time and the second preset algorithm determine the working time of the second cleaning tool.
  • the second preset algorithm is t 1 is the working time of the second cleaning tool, t s-max is the maximum continuous working time of the second cleaning tool, e is the base of the natural logarithm, ⁇ 1 and ⁇ 1 are the first corresponding to the driving state and the driver state, respectively.
  • control unit is specifically configured to correspond to the second preset weight parameter corresponding to the driving state and/or the driver state, the driving state and/or the driver state, and the maximum working capacity of the second cleaning tool.
  • Frequency, determining the working frequency of the second cleaning tool specifically including: according to the driving state, the driver state, the second preset weight parameter corresponding to the driving state, the second preset weight parameter corresponding to the driver state, and the second cleaning tool working frequency
  • the maximum frequency and the third preset algorithm determine the operating frequency of the second cleaning tool.
  • control unit is specifically configured to, after controlling the second cleaning tool to clean the first area according to the working time of the second cleaning tool and the working frequency of the second cleaning tool, further includes: first according to the driving state and/ Or the third preset weight parameter corresponding to the driver's state, the driving state and/or the driver's state, and the maximum duration of continuous operation of the third cleaning tool to determine the working duration of the third cleaning tool.
  • the working interval of the third cleaning tool is determined according to the driving state and/or the driver state, the fourth preset weight parameter corresponding to the driving state and the driver state, and the maximum working interval of the third cleaning tool.
  • the third cleaning tool is controlled to clean the first area according to the working time of the third cleaning tool and the working interval of the third cleaning tool
  • the second cleaning tool is controlled to clean the first area according to the working time of the second cleaning tool and the working frequency of the second cleaning tool. Remove the second type of debris.
  • control unit is specifically used for the third preset weight parameter corresponding to the driving state and/or the driver state, the driving state and/or the driver state, and the continuous operation of the third cleaning tool
  • the maximum duration is to determine the working duration of the third cleaning tool, which specifically includes: according to the driving state, the driver state, the third preset weight parameter corresponding to the driving state, the third preset weight parameter corresponding to the driver state, and the third cleaning tool continuous
  • the maximum working time and the fourth preset algorithm determine the working time of the third cleaning tool.
  • the fourth preset algorithm is t 2 is the working time of the third cleaning tool, t l-max is the maximum continuous working time of the third cleaning tool, e is the base of the natural logarithm, and ⁇ 3 and ⁇ 3 are the driving state and the driver state, respectively
  • the corresponding third preset weight parameter, ⁇ 3 + ⁇ 3 1, s 1 represents the driving state, and s 2 represents the driver's state.
  • control unit is specifically configured to perform according to the driving state and/or the driver state, the fourth preset weight parameter corresponding to the driving state and the driver state, and the maximum interval between the work of the third cleaning tool, Determine the working interval of the third cleaning tool, specifically including: according to the driving state, driver state, fourth preset weight parameter corresponding to the driving state, fourth preset weight parameter corresponding to the driver state, and the maximum interval between the work of the third cleaning tool And the fifth preset algorithm determines the working interval of the third cleaning tool.
  • the fifth preset algorithm is ⁇ t is the working interval of the third cleaning tool, ⁇ t max is the maximum working interval of the third cleaning tool, e is the base of the natural logarithm, ⁇ 4 and ⁇ 4 are the fourth preset corresponding to the driving state and the driver state respectively
  • the weight parameter, ⁇ 4 + ⁇ 4 1, s 1 represents the driving state, and s 2 represents the driver's state.
  • an automatic window cleaning device including: a processor and a memory; the memory is used to store computer execution instructions, and when the automatic window cleaning device is running, the processor executes the computer execution stored in the memory Instructions, so that the automatic window cleaning device executes the automatic window cleaning method of any one of the first aspect and the first aspect described above.
  • the embodiments of the present application also provide a computer-readable storage medium, including instructions, which, when run on a computer, cause the computer to execute the window automation system described in any one of the first aspect and the first aspect. Cleaning method.
  • the embodiments of the present application also provide a computer program product, including instructions, which when run on a computer, cause the computer to execute the automatic window cleaning method of any one of the above-mentioned first aspect and the first aspect .
  • FIG. 1 is a first structural diagram of a vehicle provided by an embodiment of this application.
  • Fig. 2 is a second structural diagram of a vehicle provided by an embodiment of the application.
  • FIG. 3 is a schematic structural diagram of a computer system provided by an embodiment of this application.
  • FIG. 4 is a schematic diagram 1 of the process of an automatic window cleaning method provided by an embodiment of the application.
  • FIG. 5 is a schematic diagram of an installation position of an in-vehicle camera provided by an embodiment of the application.
  • FIG. 6 is a schematic diagram of the second process of a method for automatically cleaning vehicle windows provided by an embodiment of the application.
  • FIG. 7 is a third schematic flow chart of a method for automatically cleaning vehicle windows according to an embodiment of the application.
  • FIG. 8 is a first structural diagram of an automatic window cleaning device provided by an embodiment of the application.
  • FIG. 9 is a second structural diagram of the automatic window cleaning device provided by the embodiment of the application.
  • the automatic window cleaning method provided by the embodiments of the present application is applied to a vehicle or other equipment with a window cleaning function (such as a cloud server).
  • the vehicle can implement the automatic window cleaning method provided in the embodiments of the present application through the components (including hardware and software) contained therein, and detect and automatically clean the sundries on the window.
  • FIG. 1 is a functional block diagram of a vehicle 100 provided by an embodiment of the present application.
  • the vehicle 100 can determine whether there are sundries on the windows according to the window images collected by the in-vehicle cameras, and then control the cleaning tools in the vehicle to perform automatic window cleaning to remove sundries on the windows.
  • the vehicle 100 may include various subsystems, such as a traveling system 110, a sensor system 120, a control system 130, a wireless communication system 140, a power supply 150, a computer system 160, and a user interface 170.
  • the vehicle 100 may include more or fewer subsystems, and each subsystem may include multiple elements.
  • each subsystem and element of the vehicle 100 may be interconnected by wire or wirelessly.
  • the traveling system 110 may include components that power the vehicle 100, such as an engine, a transmission, and the like.
  • the sensor system 120 may include several sensors that sense information about the environment around the vehicle 100.
  • the sensor system 120 may include a positioning system 121 (the positioning system may be a GPS system, a Beidou system or other positioning systems), an inertial measurement unit (IMU) 122, a radar sensor 123, a lidar 124, and a vision system. At least one of the sensor 125, the ultrasonic sensor 126 (not shown in the figure), and the sonar sensor 127 (not shown in the figure).
  • the sensor system 120 may also include sensors of the internal system of the monitored vehicle 100 (for example, an in-vehicle camera, an in-vehicle air quality monitor, a fuel gauge, an oil temperature gauge, etc.). Sensor data from one or more of these sensors can be used to detect objects and their corresponding characteristics (position, shape, direction, speed, etc.). Such detection and identification are key functions to ensure the safe operation of the vehicle 100.
  • the positioning system 121 can be used to estimate the geographic location of the vehicle 100.
  • the IMU 122 is used to sense changes in the position and orientation of the vehicle 100 based on inertial acceleration.
  • the IMU 122 may be a combination of an accelerometer and a gyroscope.
  • the radar sensor 123 may use electromagnetic wave signals to sense objects in the surrounding environment of the vehicle 100. In some embodiments, in addition to sensing the position of the object, the radar sensor 123 may also be used to sense the radial velocity of the object and/or the radar cross-sectional area RCS of the object.
  • the lidar 124 can use laser light to sense objects in the environment where the vehicle 100 is located.
  • the lidar 124 may include one or more laser sources, laser scanners, and one or more detectors, as well as other system components.
  • the visual sensor 125 may be used to capture multiple images of the surrounding environment of the vehicle 100 and multiple images of the environment in the vehicle, wherein the multiple images of the environment in the vehicle include a window image.
  • the vision sensor 125 may be a still camera or a video camera.
  • the control system 130 may control the operation of the vehicle 100 and its components.
  • the control system 130 may include various elements, such as at least one of a computer vision system 131, a route control system 132, an obstacle avoidance system 133, and a cleaning system 134.
  • the computer vision system 131 can be operated to process and analyze the images captured by the vision sensor 125 and the measurement data obtained by the radar sensor 123 in order to identify objects and/or features in the surrounding environment of the vehicle 100 and the interior of the vehicle, such as debris on the windows. Things. Objects and/or features in the surrounding environment of the vehicle 100 may include traffic signals, road boundaries, and obstacles.
  • the computer vision system 131 may use object recognition algorithms, structure from motion (SFM) algorithms, video tracking, and other computer vision technologies. In some embodiments, the computer vision system 131 may be used to map the environment, track objects, estimate the speed of objects, and so on.
  • the route control system 132 is used to determine the travel route of the vehicle 100.
  • the route control system 132 may combine data from the radar sensor 123, the positioning system 121, and one or more predetermined maps to determine the driving route for the vehicle 100.
  • the obstacle avoidance system 133 is used to identify, evaluate and avoid or otherwise cross over potential obstacles in the environment of the vehicle 100.
  • the cleaning system 134 is used to clean up the sundries on the vehicle windows.
  • the cleaning system 134 includes cleaning tools such as air conditioners, cleaning fluid sprayers, wipers and the like.
  • the cleaning system 134 may combine the window image information from the visual sensor 125 to clean up the debris on the window, so as to improve the safety of the driver in the process of driving the vehicle.
  • control system 130 may add or alternatively include components other than those shown and described. Alternatively, a part of the components shown above may be reduced.
  • the vehicle 100 may use the wireless communication system 140 to obtain required information, where the wireless communication system 140 may wirelessly communicate with one or more devices directly or via a communication network.
  • the wireless communication system 140 may use 3G cellular communication, such as CDMA, EVDO, GSM/GPRS, or 4G cellular communication, such as LTE. Or 5G cellular communication.
  • the wireless communication system 140 may use WiFi to communicate with a wireless local area network (WLAN).
  • WLAN wireless local area network
  • the wireless communication system 140 may directly communicate with the device using an infrared link, Bluetooth, or ZigBee.
  • Other wireless protocols, such as various vehicle communication systems, for example, the wireless communication system 140 may include one or more dedicated short range communications (DSRC) devices.
  • DSRC dedicated short range communications
  • the computer system 160 may include at least one processor 161 that executes instructions 1621 stored in a non-transitory computer readable medium such as a data storage device 162.
  • the computer system 160 may also be multiple computing devices that control individual components or subsystems of the vehicle 100 in a distributed manner.
  • the processor 161 may be any conventional processor, such as a commercially available central processing unit (CPU). Alternatively, the processor may be a dedicated device such as an application specific integrated circuit (ASIC) or other hardware-based processor.
  • FIG. 1 functionally illustrates a processor, a memory, and other elements in the same physical housing, those of ordinary skill in the art should understand that the processor, computer system, or memory may actually include a processor, a computer system, or a memory that can be stored in the same Multiple processors, computer systems, or memories in a physical housing, or include multiple processors, computer systems, or memories that may not be stored in the same physical housing.
  • the memory may be a hard drive, or other storage medium located in a different physical enclosure.
  • a reference to a processor or computer system will be understood to include a reference to a collection of processors or computer systems or memories that may operate in parallel, or a reference to a collection of processors or computer systems or memories that may not operate in parallel.
  • some components such as steering components and deceleration components may each have its own processor that only performs calculations related to component-specific functions .
  • the processor may be located away from the vehicle and wirelessly communicate with the vehicle.
  • some of the processes described herein are executed on a processor arranged in the vehicle and others are executed by a remote processor, including taking the necessary steps to perform a single manipulation.
  • FIG. 1 should not be construed as a limitation to the embodiments of the present application.
  • a car traveling on the road can identify debris on the window and determine a corresponding cleaning strategy, so that the vehicle can automatically clean the window.
  • each identified debris can be considered independently, and based on the respective characteristics of the debris, such as its shape, area, etc., as well as the driving state of the vehicle (such as speed), the driver's state (such as whether the driver is Fatigue), which can be used to determine the automatic window cleaning strategy of a driving car.
  • the vehicle 100 or the computing device associated with the vehicle 100 can detect and identify the vehicle window image based on the acquired window image. Debris.
  • the vehicle 100 is able to adjust its cleaning strategy based on the predicted debris on the windows and the driving state of the vehicle and the driver state. In other words, the vehicle can determine when and how often the cleaning tools in the vehicle will need to work based on the predicted debris. In this process, other factors may also be considered to determine the automatic window cleaning strategy of the vehicle 100, such as the state of the surrounding vehicles and the weather conditions during the driving of the vehicle 100.
  • the computing device can also provide instructions for the vehicle 100 to automatically clean the windows and debris, so that the car is in the process of driving. Under the condition of ensuring safety, automatically clean the sundries on the windows (for example, the air conditioner blows hot/cold air, sprays cleaning fluid, and swings the wiper) to keep the windows clean.
  • the air conditioner blows hot/cold air, sprays cleaning fluid, and swings the wiper
  • the above-mentioned vehicle 100 may be a car, a truck, a motorcycle, a bus, a boat, an airplane, a helicopter, a recreational vehicle, a construction equipment, a tram, a golf cart, and a train, etc.
  • the embodiment of the present application does not specifically limit it.
  • the vehicle may include the following modules:
  • the environment perception module 201 is used to obtain the car window image captured by the camera in the car, combine the car window image to determine whether there are sundries on the car window, including the first type of sundries and the second type of sundries, and determine the second type of sundries The location of the object.
  • the environment sensing module 201 at least includes detection devices such as a temperature sensor and a camera. Real-time acquisition of information such as window images and window videos through the camera in the car, real-time acquisition of information such as temperature data inside and outside the car through the temperature sensor, and transfer of the acquired information to the central processing module 202 so that the central processing module 202 can generate Corresponding automatic window cleaning strategy.
  • the central processing module 202 (for example, a vehicle-mounted computer) is used to receive information such as window images, window videos, and temperature data inside and outside the vehicle from the environment perception module 201.
  • the central processing module 202 combines the current vehicle driving state (such as speed ) And the driver's status (such as whether the driver is fatigued, whether the driver is distracted), detect and analyze the window image and window video received from the environment perception module 201, and determine the presence of the first type of debris on the window And/or the second type of debris, and generate the corresponding cleaning decision (such as the working time and frequency of the wiper, etc.), output the action instruction corresponding to the cleaning decision, and send the action instruction to the action execution module 203 to instruct the action
  • the execution module 203 automatically cleans the vehicle windows according to the action instructions.
  • the action execution module 203 is configured to receive action instructions from the central processing module 202 and complete the automatic window cleaning operation according to the action instructions.
  • the action execution module includes at least cleaning tools in the vehicle, such as wipers, air conditioners, and cleaning liquid sprayers.
  • In-vehicle communication module 204 (not shown in FIG. 2): used for information exchange between the own vehicle and other vehicles.
  • the storage component 205 (not shown in FIG. 2) is used to store the executable codes of the above-mentioned modules. Running these executable codes can implement part or all of the method procedures in the embodiments of the present application.
  • the computer system 160 shown in FIG. 1 includes a processor 301, and the processor 301 is coupled to a system bus 302.
  • the processor 301 may be one or more processors, where each processor may include one or more processor cores.
  • a display adapter (video adapter) 303, the display adapter 303 can drive the display 309, and the display 309 is coupled to the system bus 302.
  • the system bus 302 is coupled with an input/output (I/O) bus (BUS) 305 through a bus bridge 304.
  • the I/O interface 306 and the I/O bus 305 are coupled.
  • the I/O interface 306 communicates with various I/O devices, such as an input device 307 (such as a keyboard, a mouse, a touch screen, etc.), and a media tray 308 (such as a CD-ROM, a multimedia interface, etc.).
  • the transceiver 315 can send and/or receive radio communication signals
  • the camera 310 can capture static and dynamic digital video images
  • USB universal serial bus
  • the interface connected to the I/O interface 306 may be a USB interface.
  • the processor 301 may be any traditional processor, including a reduced instruction set computer (RISC) processor, a complex instruction set computer (CISC) processor, or a combination of the foregoing.
  • the processor may be a dedicated device such as an application specific integrated circuit (ASIC).
  • the processor 301 may be a neural network processor or a combination of a neural network processor and the foregoing traditional processors.
  • the computer system 160 may be located away from the vehicle and may communicate wirelessly with the vehicle 100.
  • some of the processes described herein may be configured to be executed on a processor in the vehicle, and other processes may be executed by a remote processor, including taking actions required to perform a single manipulation.
  • the computer system 160 may communicate with a software deployment server (deploying server) 313 through a network interface 312.
  • the network interface 312 is a hardware network interface, such as a network card.
  • the network 314 may be an external network, such as the Internet, or an internal network, such as an Ethernet or a virtual private network (VPN).
  • the network 314 may also be a wireless network, such as a WiFi network, a cellular network, and so on.
  • the automatic window cleaning method of the embodiments of the present application may also be executed by a chip system.
  • the embodiment of the application provides a chip system.
  • the main CPU (Host CPU) and the neural network processor (neural processing unit, NPU) work together to implement the corresponding algorithm for the functions required by the vehicle 100 in Figure 1, and the corresponding algorithm for the functions required by the vehicle shown in Figure 2 ,
  • the corresponding algorithm for the functions required by the computer system 160 shown in FIG. 3 can also be implemented.
  • the computer system 160 may also receive information from other computer systems or transfer information to other computer systems.
  • the sensor data collected from the sensor system 120 of the vehicle 100 can be transferred to another computer, and the data can be processed by the other computer.
  • the data from the computer system 160 may be transmitted to the computer system on the cloud side via the network for further processing.
  • the network and intermediate nodes can include various configurations and protocols, including the Internet, World Wide Web, Intranet, virtual private network, wide area network, local area network, private network using one or more company’s proprietary communication protocols, Ethernet, WiFi and HTTP, And various combinations of the foregoing. This communication can be performed by any device capable of transferring data to and from other computers, such as modems and wireless interfaces.
  • the automatic window cleaning method provided by the embodiments of the present application is applied in the scene of window cleaning, and can be executed by the control chip, processor, etc. in the vehicle, such as the processor 161 in FIG. 1 or the processor 301 in FIG. 3,
  • the automatic window cleaning method provided by the embodiment of the present application may also be executed by other devices with a window cleaning function, such as a cloud server.
  • the automatic window cleaning method according to the embodiment of the present application will be described in detail below with reference to the accompanying drawings.
  • the present application provides an automatic cleaning method for vehicle windows.
  • the method includes the following steps S401-S405.
  • the embodiments of the present application will be described below with reference to FIGS. 4 and 5:
  • the vehicle window referred to in the embodiment of the present application includes a front windshield, a rear windshield, and a side window glass.
  • the window image refers to the image of the front windshield of the vehicle and/or the image of the rear windshield of the vehicle.
  • the car window image may also be a side window glass image of the vehicle.
  • the window image does not include other parts of the car body.
  • a car window image can be obtained by shooting with a camera in the car.
  • the number of cameras in the car can be one or more.
  • the camera needs to be set at a position where the image of the car window can be obtained.
  • the area 501 represents the front windshield of the vehicle
  • the area 502 represents the rear windshield of the vehicle
  • the areas 503 and 504 represent the window glass on the side of the vehicle.
  • the in-vehicle camera can be set at the position where the rear windshield of the vehicle is connected to the top of the vehicle, such as the positions shown in 506 and 508. The camera is located at this position to obtain the image of the rear windshield of the vehicle.
  • the camera in the car can also be set at the position where the window glass on the side of the vehicle is connected to the top of the car, such as the positions shown in 505 and 509. The camera is located at this position to obtain the image of the front windshield of the vehicle and the rear of the vehicle.
  • the in-vehicle camera can also be set on the top of the vehicle, such as the position shown in 507, where the camera is located to obtain the image of the front windshield of the vehicle, the rear windshield image of the vehicle, or the image of the window glass on the side of the vehicle.
  • the position of the camera in the vehicle can be determined by itself according to requirements, and is not limited to the position shown in FIG. 5.
  • the camera may also be placed at a position where the front windshield of the vehicle is connected to the inner roof of the vehicle, and the camera is located at this position to obtain an image of the front windshield of the vehicle.
  • the position of the camera in the car must be at least set to a position where the front windshield of the vehicle (especially the front windshield on the driver's side of the vehicle) can be photographed in the car.
  • the camera can take real-time images of the front windshield of the vehicle, so as to detect whether there is debris on the front windshield in real time.
  • a single frame of the car window image can be obtained by a single shot, or multiple frames of the car window image can be obtained by multiple consecutive shots.
  • the embodiment of the present application can detect whether there are first-type sundries, such as fog, frost, etc., on the vehicle window according to a single frame of the vehicle window image.
  • the embodiment of the present application may detect whether there are second-type debris, such as dust, fallen leaves, etc., on the vehicle window based on the multi-frame vehicle window image.
  • S402 Determine a dark channel image corresponding to a single frame of car window image.
  • the single frame of the window image is obtained by the camera in a single shot in the above step S401, or it may be the last frame of the multiple frames of the window image obtained by the camera in the above step S401 continuously shooting multiple times, or it may be the above step In S401, any one of the multiple frames of vehicle window images obtained by the camera continuously shooting multiple times.
  • the dark channel image is a gray value image of a single frame of car window image. Compared with a single frame of car window image, its gray value image can more clearly reflect whether there is fog on the car window. Therefore, in the embodiment of the present application, the dark channel image of a single frame of the car window is used to detect whether there is fog on the car window.
  • each pixel of the dark channel image is based on the value of the corresponding pixel in the single frame window image at the same position. Calculated.
  • I represents a single frame of car window image
  • c represents the color channel of the pixel, including r channel, g channel, and b channel, namely red channel, green channel and blue channel
  • I c (x) represents pixel x in r channel
  • w is the set of pixels in the pixel area of a*a centered on pixel x in a single frame of car window image
  • a is an integer greater than 0, a
  • the value of can be determined according to actual needs, and the more common values of a are 1, 3, 5, 7, and so on.
  • J dark represents the gray value of the pixel x
  • the value of J dark is the minimum value of the pixel in w on the color channel (including the three color channels of r, g, and b).
  • a single frame of car window image includes 5*5 pixels
  • pixel x is the pixel in the third row and third column of the single frame of car window image.
  • w is a set of pixels in a 3*3 pixel area centered on pixel x in a single frame of car window image, then the pixel set w includes the second row, second column, and second row of the single frame car window image. 3 columns, 2nd row 4th column, 3rd row 2nd column, 3rd row 3rd column, 3rd row 4th column, 4th row 2nd column, 4th row 3rd column, and 4th row 4 columns of pixels.
  • the RGB values of these pixels are (255,146,85), (254,150,90), (240,165,100), (244,163,98), (240,159,95), (248, 160, 99), (249, 159, 89), (239, 149, 85), and (245, 157, 99).
  • the minimum value of the 3*3 pixels in w on the three color channels of r channel, g channel and b channel is 85, then the gray value of the pixel x to be processed is 85, that is The RGB value of the pixel x corresponding to the dark channel image is (85, 85, 85).
  • the number of pixels in the pixel set w in the pixel area with the size a*a centered on the edge pixel x is less than or equal to a*a.
  • a single frame of car window image includes 5*5 pixels, and pixel x is the pixel in the fifth row and fifth column of the single frame of car window image.
  • w is a set of pixels in a 3*3 pixel area centered on pixel x in a single frame of car window image, then the pixel set w includes the fourth row, fourth column, and fourth row of the single frame car window image. Pixels in 5 columns, 5th row and 4th column, and 5th row and 5th column.
  • the RGB values of these pixels are (245, 157, 99), (255, 159, 90), (244, 163, 95), and (254, 165, 89), respectively.
  • the minimum value of the four pixels in w on the three color channels of the r channel, the g channel and the b channel is 89, then the gray value of the pixel x is 89, that is, the pixel x is dark
  • the RGB value of the corresponding pixel in the channel image is (89,89,89).
  • S403 According to the number of pixels in the dark channel image whose gray value exceeds a preset gray threshold and/or the clarity of the dark channel image, determine that there is a first type of debris on the vehicle window.
  • the first type of debris is debris such as fog or frost.
  • the presence of the first type of debris on the car window can be determined based on the number of pixels in the dark channel image whose gray value exceeds the preset gray threshold, or based on the clarity of the dark channel image, or a combination of these two factors determine. Therefore, there are three ways to determine the existence of the first type of debris on the car window, as shown in the following (1)-(3):
  • the corresponding dark channel image is the gray value image of the single frame of car window image
  • the dark channel image contains 10*10 pixels
  • the gray value of the pixels ranges from 0 to 255.
  • the preset gray threshold is 100, the number of pixels in the dark channel image whose gray value does not exceed the preset gray threshold is 30, and the number of pixels whose gray value exceeds the preset gray threshold is 70, 30 ⁇ 70 , So there is the first type of debris on the window; if the preset gray threshold is 100, the number of pixels in the dark channel image whose gray value does not exceed the preset gray threshold is 50, and the gray value exceeds the preset gray The number of pixels with the degree threshold is 50, therefore, there are first-type debris on the car window; if the preset gray threshold is 100, the number of pixels in the dark channel image whose gray value does not exceed the preset gray threshold is 70. The number of pixels whose gray value exceeds the preset gray threshold is 30, and 70>30. Therefore, there is no first type of debris on the car window.
  • the corresponding dark channel image is the gray value image of the single frame of car window image
  • the dark channel image contains 10*10 pixels
  • the grayscale image of is corresponding to the pixels in the single-frame car window image, and the grayscale value of the pixels ranges from 0 to 255. Taking the gray value of the pixel as the abscissa and the number of pixels with the same gray value as the ordinate, the gray value histogram of the dark channel image is established.
  • the definition of the dark channel image is the variance of the gray values of pixels in the dark channel image.
  • the corresponding dark channel image contains 2*2 pixels (the 2*2 pixels are A, B, C, D)
  • the grayscale value of the pixel ranges from 0 to 255.
  • the gray value of pixel A is 7
  • the gray value of pixel B is 1
  • the gray value of pixel C is 3
  • the gray value of pixel D is 5, pixels A, B, C, D
  • the preset resolution is 6, the first type of debris is present on the window; if the preset resolution is 5, then there is no first type of debris on the window; if the preset resolution is 4, then the window The first type of debris does not exist on it.
  • the definition of the dark channel image is determined according to the information entropy of the pixels in the dark channel image.
  • the information entropy of the dark channel image passes To determine, H is the information entropy of the dark channel image, d represents the gray value of the pixel, n represents the maximum value of the gray value of the pixel, and p d represents the probability of the pixel with the gray value d in the dark channel image.
  • the information entropy of the dark channel image can indicate the gray distribution in the image. The greater the information entropy of the dark channel image, the greater the range of the gray value of the pixels in the dark channel image, and the higher the clarity of the image.
  • the definition of the dark channel image is the saturation of the dark channel image obtained by calculation using an HSV (hue, saturation, value) color model.
  • HSV hue, saturation, value
  • the measurement method used to determine or represent the sharpness of an image is not limited to the information entropy and saturation given in the embodiment of this application, and the hue and value in the HSV color model. It can also be used to determine or indicate the sharpness of an image, and the specific measurement of the sharpness of an image can be determined according to actual applications.
  • the number of pixels in the dark channel image whose gray value does not exceed the preset gray threshold is less than or equal to the number of pixels in the dark channel image whose gray value exceeds the preset gray threshold, and the sharpness of the dark channel image is less than the preset clear Degree, it is determined that the first type of debris is present on the window.
  • the dark channel image as the variance of the pixels in the dark channel image as an example
  • a single frame of car window image contains 2*2 pixels
  • the corresponding dark channel image contains 2*2 A grayscale image of pixels (the 2*2 pixels are A, B, C, and D respectively), and the grayscale value of the pixel ranges from 0 to 255.
  • the gray value of pixel A is 7
  • the gray value of pixel B is 1
  • the gray value of pixel C is 3,
  • the gray value of pixel D is 5,
  • the preset gray threshold is 2
  • the preset resolution is 6, the gray value of pixel B does not exceed the preset gray threshold, and the gray value of pixels A, C, D exceeds the preset gray threshold, 1 ⁇ 3
  • the definition of the dark channel image is 5 ⁇ 6, then there is the first type of debris on the window;
  • the preset gray threshold Is 2 the preset resolution is 4, the gray value of pixel B does not exceed the preset gray threshold, the gray value of pixels A, C, D exceeds the preset gray threshold, 1 ⁇ 3, the dark channel image is clear If the degree 5>4, the first type of debris does not exist on the window; if the preset gray threshold is 5, the preset resolution is 6, and the gray value of pixels
  • consecutive i frames of car window images are part of the car window images in the multiple frames of car window images captured by the camera in step S401, where i is an integer greater than 1.
  • the second type of debris may be impurities such as dust, fallen leaves, bird droppings and the like.
  • the RGB values of the pixels in the consecutive i frames of the car window image that is, the values of the pixels in the consecutive i frames of the car window image on the three color channels of r, g, and b, determine that the same in the consecutive i frames of the car window image
  • the position of the pixel's RGB value changes, and the image background model is established.
  • the pixels in the image background model represent the changes in the RGB values of the pixels in the consecutive i frames of the car window image.
  • the RGB value of the pixel at the same position in the consecutive i frames of the car window image has not changed, then in the image background model, the RGB value of the pixel corresponding to the pixel that has not changed in the consecutive i frame of the car window image (0, 0, 0); if the RGB value of the pixel at the same position in the consecutive i frames of the car window image changes, the RGB of the pixel corresponding to the pixel that changes in the consecutive i frame of the car window image in the image background model
  • the value is an RGB value other than (0, 0, 0).
  • the RGB value in the continuous i-frame window image can be determined according to the RGB value of the pixel in the image background model Pixels that have not changed, or pixels whose RGB values in consecutive i-frames of car window images do not change to a degree that do not exceed the preset change threshold, correspond to the window area corresponding to the first area, and there are second-type debris in the first area.
  • each frame of a car window image includes 2*2 pixels.
  • the first frame of car window image includes pixels A1, B1, C1, and D1
  • the RGB values of these 4 pixels are (1, 2, 3), (3, 5, 6), (1, 9, 5) and (2, 3, 0).
  • the second frame of the car window image includes pixels A2, B2, C2 and D2, the RGB values of these 4 pixels are (3, 2, 5), (7, 5, 2), (7, 1, 9) and (8, 5, 2), respectively.
  • the 3 frames of car window images include pixels A3, B3, C3 and D3.
  • the RGB values of these 4 pixels are (3, 2, 1), (5, 5, 4), (5, 1, 7) and ( 6, 3, 2).
  • RGB values of A4, B4, C4 and D4 are (4/3, 0, 8/3), (8/3, 0, 8/3), (4, 16/3, 8/3) and (4 , 4/3, 4/3).
  • the preset change threshold is 9, for the pixels A4, B4, C4, and D4 in the image background model corresponding to the two consecutive frames of car window images, 4/3+4/3 ⁇ 9, 8/3+8 /3 ⁇ 9, 4+16/3+8/3>9, 4+4/3+4/3 ⁇ 9, that is, the continuous 3 frames of car window images corresponding to pixels A4, B4 and D4 in the image background model
  • the window area corresponding to pixels A4, B4, and D4 in the image background model that is, the pixels in the continuous 3 frames of the window image
  • the window area corresponding to A1, B1, and D1 corresponds to the first area, and there are second-type debris in the first area.
  • an image background model when using the RGB values of pixels in consecutive i frames of car window images to establish an image background model, first determine the mean value of the RGB values of pixels at the same position in the consecutive i frames of car window images, and then according to the obtained The average value of the RGB values of the pixels at each position in the continuous i frames of car window images is determined to determine the average image corresponding to the continuous i frames of car window images. According to the difference between the RGB value of the pixel in the average image and the RGB value of the pixel at the same position in the i-th frame of the car window image, an image background model is established.
  • each frame of car window image includes 2*2 pixels.
  • the first frame of car window image includes pixels A1, B1, C1, and D1
  • the RGB values of these 4 pixels are (1, 2, 3), (3, 5, 6), (1, 9, 5) and (2, 3, 0).
  • the second frame of the car window image includes pixels A2, B2, C2 and D2, the RGB values of these 4 pixels are (3, 2, 5), (7, 5, 2), (7, 1, 9) and (8, 5, 2), respectively.
  • the RGB values of the pixels at the same position in two consecutive frames of car window images determine that the RGB values of pixels A3, B3, C3, and D3 in the average image are (2, 2, 4), (5, 5, 4), respectively , (4, 5, 7) and (5, 4, 1), the difference between the RGB value of the pixel in the average image and the RGB value of the pixel in the second frame of the car window image is (1, 0, 1), (2,0,2), (3,4,2) and (3,1,1), the image background model is obtained according to the two consecutive frames of car window images.
  • the pixels A4, B4, C4 in the image background model The RGB values of and D4 are (1, 0, 1), (2, 0, 2), (3, 4, 2) and (3, 1, 1), respectively.
  • the first preset algorithm is a saliency detection algorithm or a target detection algorithm.
  • each frame of a car window image includes 2*2 pixels.
  • the first frame of car window image includes pixels A1, B1, C1, and D1
  • the second frame of the car window image includes pixels A2, B2, C2, and D2
  • the image background model corresponding to the two consecutive frames of the car window image includes pixels A3, B3, C3, and D3.
  • the saliency detection algorithm or the target detection algorithm it is determined that the RGB values of the pixels C1 and D1 and C2 and D2 in the continuous two frames of the car window image corresponding to the pixels C3 and D3 in the image background model have not changed, so the image background model
  • the window area corresponding to the pixels C3 and D3, that is, the window area corresponding to the pixels C1 and D1 and C2 and D2 in the continuous two frame window images is the first area, and the second type of debris exists in the first area.
  • steps S402 to S403 are used to detect the first type of sundries on the window
  • step S404 is used to detect whether there is a second type of sundries on the window.
  • the order of detecting whether there is a first type of sundries on the window and detecting whether there is a second type of sundries on the window is not limited, that is, step S402 and step S403 can be performed first, and then step S404 is performed, or it can be performed first Step S404, then perform step S402 and step S403, or perform step S402, step S403 and step S404 at the same time.
  • the cleaning tool includes a first cleaning tool and a second cleaning tool.
  • the first cleaning tool is mainly used to remove the first type of debris, such as air conditioners, through operations such as dehumidification and temperature adjustment.
  • the second cleaning tool is mainly used to remove raindrops and dust attached to the windshield of the vehicle to improve the visibility of the driver, such as windshield wipers.
  • the specific implementation methods for removing the first type of debris and the second type of debris are respectively introduced below.
  • the first cleaning tool is controlled to be turned on for a preset time period according to the temperature data inside and outside the vehicle to remove the first type of debris.
  • the preset time period may be determined based on the temperature data inside and outside the vehicle, or may be determined by the driver himself.
  • the air conditioner blowing temperature is set to T2 and the air conditioner blowing duration is set according to the interior and exterior temperature data S1, the air conditioner is turned on at the blowing temperature T2, and turned off after the opening time S1, to reduce the temperature of the car windows and clean the fog on the windows;
  • the air conditioning temperature is T2
  • step S401 to step S403 are performed again to detect whether the first type of debris still exists on the car window, and if so, the cleaning tool is re-controlled to remove the first type of debris .
  • the second cleaning tool is controlled to work at a preset operating frequency for a preset period of time, so as to clean the second type of debris in the first area.
  • the preset working frequency and working duration of the second cleaning tool can be preset according to the cleaning effect of the second cleaning tool, or can be set by the driver himself.
  • adaptive adjustments to the working time and frequency of the second cleaning tool can minimize the impact of window cleaning on the driver and ensure that the vehicle is driving Security.
  • the second cleaning tool when controlling the second cleaning tool to clean the second type of debris in the first area, it may first be based on the driving state and/or the driver state, the driving state and /Or the first preset weight parameter corresponding to the driver's state and the maximum duration of continuous operation of the second cleaning tool to determine the working duration of the second cleaning tool. Then determine the operating frequency of the second cleaning tool according to the driving status and/or the driver status, the second preset weight parameter corresponding to the driving status and/or the driver status, and the maximum frequency of the second cleaning tool, and finally according to the second The working time of the cleaning tool and the working frequency of the second cleaning tool control the second cleaning tool to clean the first area and remove the second type of debris.
  • the above-mentioned driving state may be the driving speed of the vehicle, including high speed, medium speed, and low speed
  • the above-mentioned driver state may be the degree of fatigue of the driver, including severe fatigue, light fatigue, and no fatigue, which will not be described in detail below.
  • first preset weight parameter and the second preset weight parameter may be preset, or may be set by the driver himself.
  • the working time of the second cleaning tool may be determined according to the driving state, may also be determined according to the driver's state, or may be determined in combination with these two factors.
  • the driver state the first preset weight parameter corresponding to the driving state
  • the first preset weight parameter corresponding to the driver state and the second preset algorithm Determine the working time of the second cleaning tool.
  • t 1 is the working time of the second cleaning tool
  • t s-max is the maximum continuous working time of the second cleaning tool
  • e is the base of the natural logarithm
  • ⁇ 1 and ⁇ 1 are respectively corresponding to the driving state and the driver state
  • the first preset weight parameter, ⁇ 1 + ⁇ 1 1, s 1 represents the driving state
  • s 2 represents the driver's state.
  • the values of the driving state s 1 are -1, -3, and -10, which respectively represent driving speeds of high speed, medium speed, and low speed
  • the values of the driver state s 2 are -1, -3, -10 , Respectively represent the status of the driver as severe fatigue, mild fatigue, and non-fatigue.
  • the greater the value of s 1 the greater the driving speed, or the greater the value of s 2 , the more fatigued the driver, and the shorter the working time t 1 of the second cleaning tool.
  • the first preset weight parameter corresponding to the driving state and the second preset algorithm Determine the working time of the second cleaning tool, where t 1 is the working time of the second cleaning tool, t s-max is the maximum continuous working time of the second cleaning tool, e is the base of the natural logarithm, and ⁇ 1 is the corresponding driving state
  • the working frequency of the second cleaning tool can be determined according to the driving state, can also be determined according to the driver's state, or can be determined in combination with these two factors.
  • f is the working frequency of the second cleaning tool
  • f max is the maximum frequency of the second cleaning tool
  • e is the base of the natural logarithm
  • ⁇ 2 and ⁇ 2 are the second presets corresponding to the driving state and the driver state, respectively
  • the weight parameter, ⁇ 2 + ⁇ 2 1, s 1 represents the driving state, and s 2 represents the driver's state.
  • the values of the driving state s 1 are -1, -3, and -10, which respectively represent driving speeds of high speed, medium speed, and low speed
  • the values of the driver state s 2 are -1, -3, -10 , Respectively represent the status of the driver as severe fatigue, mild fatigue, and non-fatigue.
  • the greater the value of s 1 the greater the driving speed, or the greater the value of s 2 , the more fatigued the driver is, and the greater the operating frequency f of the second cleaning tool.
  • the second preset weight parameter corresponding to the driving state and the third preset algorithm Determine the operating frequency of the second cleaning tool.
  • f is the working frequency of the second cleaning tool
  • f max is the maximum frequency of the second cleaning tool
  • e is the base of the natural logarithm
  • ⁇ 2 is the second preset weight parameter corresponding to the driving state
  • ⁇ 2 1
  • s 1 represents the driving state
  • the value of s 1 refers to the above.
  • the second preset weight parameter corresponding to the driver state and the third preset algorithm Determine the operating frequency of the second cleaning tool.
  • f is the working frequency of the second cleaning tool
  • f max is the maximum frequency of the second cleaning tool
  • e is the base of the natural logarithm
  • ⁇ 2 is the second preset weight parameter corresponding to the driver state
  • ⁇ 2 1
  • s 2 represents the state of the driver
  • the value of s 2 refers to the above.
  • the value of the driving state s 1 and the value of the driver state s 2 may refer to the foregoing.
  • t 1 is longer and f is smaller, which can reduce the first Second, the discomfort caused by the excessive frequency of the cleaning tool, and the working time of the second cleaning tool becomes longer, so the cleaning efficiency of the car window can be ensured.
  • the cleaning tool needs to be controlled to remove the first type of debris and the second type of debris.
  • the order of controlling the cleaning tool to remove the first type of debris and controlling the cleaning tool to remove the second type of debris is not limited.
  • the cleaning tool can be controlled to remove the first type of debris, and then the cleaning tool can be controlled to remove the second type of debris. It is also possible to first control the cleaning tool to remove the second type of debris, and then control the cleaning tool to remove the first type of debris, or it is also possible to control the cleaning tool to remove the first type of debris and the second type of debris at the same time.
  • controlling the cleaning tool to remove the first type of debris and controlling the cleaning tool to remove the second type of debris can refer to the implementations described in (1) and (2) above in this step.
  • the gray value of the pixel in the dark channel image corresponding to a single frame of the window image is used to detect whether there are first-type debris on the window , And/or use the RGB values of the pixels in the consecutive i frames of the window image to detect whether there is a second type of debris on the window. If there are first-type debris and/or second-type debris on the window, control the cleaning tool to remove the debris.
  • cleaning after detection of debris can reduce blind cleaning, improve cleaning efficiency, and save cleaning resources.
  • the embodiments of the present application can realize automatic cleaning of the vehicle windows, reduce manual operations, and improve safety during driving of the vehicle.
  • this application also provides an automatic window cleaning method.
  • the method further includes the following step S601 -S602, the embodiment of the present application will be described below in conjunction with FIG. 6:
  • S601 Determine a second region corresponding to the first region on each frame of the car window image in the continuous j frames of car window images, to obtain j second regions.
  • the continuous j frames of window images are part of the window images in the multi-frame window images captured by the camera in the above step S401, or the continuous j frames of window images are the multiple frames obtained after the camera has captured multiple times.
  • Part or all of the window images in the window image, j is an integer greater than 1, and the continuous i-frame window image and the continuous j-frame window image may include at least one other window image, or may not include others Frame of car window image.
  • the window area corresponding to all pixels in the second area is the first area.
  • each frame of the car window image includes 2*2 pixels.
  • the first frame of the car window image includes pixels A1, B1, C1, and D1
  • the second frame of the car window image includes pixels A2, B2, C2, and D2.
  • the second area corresponding to the first area in the first frame of the car window image is E1, and E1 includes pixels A1 and B1, and the RGB values of these two pixels are (1, 2, 3) and (3, 5, 6).
  • the second area corresponding to the first area in the second frame of the car window image is E2, and E2 includes pixels A2 and B2.
  • the RGB values of these two pixels are (3, 2, 5) and (7, 5, 2).
  • the third region corresponding to the first region in the image background model corresponding to the consecutive i frames of the car window image may be determined first , And then determine the difference between the RGB value of each pixel in the second area and the RGB value of the pixel at the same position in the third area, and accumulate the obtained difference to obtain the accumulated value. If the accumulated value exceeds the preset threshold, it is determined that the RGB value of the pixel at the same position in the j second area has changed; if the accumulated value does not exceed the preset threshold, the RGB value of the pixel at the same position in the j second area is determined No changes have occurred.
  • the first second area includes pixel A1 and pixel B1, and the RGB values of these two pixels are (1, 2, 3) and (3, 5, 6), respectively.
  • the second second area includes pixel A2 and pixel B2, and the RGB values of these two pixels are (3, 2, 5) and (7, 2, 5), respectively.
  • the third area includes pixels A3 and B3, the RGB values of these two pixels are (2, 0, 2) and (4, 3, 1) respectively.
  • the accumulated value of the difference between the RGB value of each pixel in the second area and the RGB value of the pixel at the same position in the third area is determined, where, F 0 represents the third area, f k represents the second area corresponding to the k-th frame of the window image of the continuous j frames of window images, k is an integer greater than 0,
  • ⁇ is the distance measurement function, in this example Among them, the measurement algorithm is an absolute value algorithm.
  • the absolute value of the difference between the RGB values of the pixels A1 and A2 in the second area and the RGB values of the pixel A3 in the third area can be determined and be (1,2,1) and (1,2,5), respectively.
  • the differences between the RGB values of the pixels B1 and B2 in the second area and the pixel B3 in the third area are (1, 2, 5) and (3, 1, 4), respectively. These differences are accumulated,
  • the accumulated value h is (6, 7, 15).
  • the image background model corresponding to the consecutive i frames of the car window image may be determined first
  • the third area corresponding to the first area is determined, and the fourth area corresponding to the first area in the image background model corresponding to the consecutive j frames of window images is determined, and the RGB values of the pixels in the third area are calculated and the first area is calculated.
  • the difference between the RGB values of the pixels in the four regions is accumulated by accumulating the obtained difference to obtain an accumulated value.
  • the accumulated value exceeds the preset threshold, it is determined that the RGB value of the pixel at the same position in the j second area has changed; if the accumulated value does not exceed the preset threshold, the RGB value of the pixel at the same position in the j second area is determined No changes have occurred.
  • the first second area includes pixel A1 and pixel B1, and the RGB values of these two pixels are (1, 2, 3) and (3, 5, 6), respectively.
  • the second second area includes pixel A2 and pixel B2, and the RGB values of these two pixels are (3, 2, 5) and (7, 3, 6), respectively.
  • the fourth area includes pixels A4 and B4.
  • the RGB values of these two pixels are (1, 0, 1) and (2, 1, 0).
  • h ⁇ k
  • ⁇ is the distance measurement function, in this example,
  • the measurement algorithm is the Hamming distance algorithm.
  • the RGB value of the pixel A3 in the third area is the same as the RGB value of the pixel A4 in the fourth area, then the Hamming distance between the pixel A3 in the third area and the pixel A4 in the fourth area is 0, and the pixel A4 in the third area
  • ⁇ may also be the Euclidean distance algorithm.
  • the RGB values of the pixels at the same position in the j second regions when it is determined that the RGB values of the pixels at the same position in the j second regions have changed, it may be determined first that the mean image corresponding to the consecutive i frames of the car window image and The fifth area corresponding to the first area. The difference between the RGB value of each pixel in the second area and the RGB value of the pixel at the same position in the fifth area is determined, and the difference is accumulated to obtain an accumulated value. If the accumulated value exceeds the preset threshold, it is determined that the RGB value of the pixel at the same position in the j second area has changed; if the accumulated value does not exceed the preset threshold, the RGB value of the pixel at the same position in the j second area is determined No changes have occurred.
  • the j second areas are determined according to the consecutive j frames of window images after the consecutive i frames of window images, and the j second areas are determined according to the jth window images.
  • it is verified whether there are second-type sundries in the first area, so as to improve the accuracy of determining the second-type sundries on the vehicle windows, and further improve the efficiency of automatic cleaning of the vehicle windows.
  • this application also provides an automatic cleaning method for car windows.
  • the cleaning tool may also include a third cleaning tool, where the third cleaning tool is mainly used for cleaning vehicle windows, such as glass water. Therefore, after the second cleaning tool is used to clean the second type of debris in the first area according to the specific implementation (2) and specific implementation (3) of step 405, the present application can also be combined with a third cleaning tool
  • the method further includes steps S701-S703. The following describes an embodiment of the present application with reference to FIG. 7:
  • S701 Determine whether the second-type sundries in the first area on the vehicle window have been cleaned.
  • the first area is the first area determined in step S404, that is, the first area in the method described in FIG. 6.
  • step S701 for the specific implementation manner of this step S701, reference may be made to the real-time manner described in step S404 in the foregoing embodiment.
  • step S701 is executed to determine whether the second-type debris in the first area on the vehicle window is cleaned. If the second type of debris in the first area on the car window has been cleaned, the automatic window cleaning process is ended, and the camera in the car is kept on to monitor the car window in real time; if the car window is in the first area The second type of debris is not cleaned (that is, the second type of debris still exists in the first area on the window), then the following step S702 is executed.
  • the third cleaning tool can be controlled to work at a preset working interval for a preset time to clean the second type of debris in the first area.
  • the preset working interval and the preset working duration of the third cleaning tool can be preset according to the cleaning effect of the third cleaning tool, or can be set by the driver himself.
  • the third cleaning tool may be a cleaning liquid, and the working time of the third cleaning tool is the spraying time of the cleaning liquid.
  • adaptive adjustments to the working time and working interval of the third cleaning tool can minimize the impact of window cleaning on the driver and ensure that the vehicle is driving. Security. Therefore, in a possible implementation manner, when controlling the third cleaning tool to clean the second type of debris in the first area, it may first be based on the driving state and/or the driver state, the driving state and/or the driver state.
  • the corresponding third preset weight parameter and the maximum duration of continuous working of the third cleaning tool determine the working duration of the third cleaning tool.
  • the working interval of the third cleaning tool is determined according to the driving state and/or the driver state, the fourth preset weight parameter corresponding to the driving state and/or the driver state, and the maximum working interval of the third cleaning tool.
  • the third cleaning tool is controlled to clean the first area according to the working time of the third cleaning tool and the working interval of the third cleaning tool, and then the second cleaning tool is controlled to clean the first area according to the working time of the second cleaning tool and the working frequency of the second cleaning tool.
  • the third cleaning tool may be a cleaning liquid
  • the working time of the third cleaning tool is the spraying time of the cleaning liquid.
  • the above-mentioned driving state may be the driving speed of the vehicle, including high speed, medium speed, and low speed
  • the above-mentioned driver state may be the fatigue degree of the driver, including severe fatigue, light fatigue, and no fatigue, which will not be described in detail below.
  • the third preset weight parameter and the fourth preset weight parameter may be preset, or may be set by the driver himself.
  • the working time of the third cleaning tool may be determined according to the driving state, may also be determined according to the driver's state, or may be determined in combination with these two factors.
  • the driver state the third preset weight parameter corresponding to the driving state
  • the third preset weight parameter corresponding to the driver state and the fourth preset algorithm Determine the working time of the third cleaning tool.
  • t 2 is the working time of the third cleaning tool
  • t l-max is the maximum continuous working time of the third cleaning tool
  • e is the base of the natural logarithm
  • ⁇ 3 and ⁇ 3 are respectively corresponding to the driving state and the driver state
  • the third preset weight parameter, ⁇ 3 + ⁇ 3 1, s 1 represents the driving state, and s 2 represents the driver's state.
  • the values of the driving state s 1 are -1, -3, and -10, which respectively represent driving speeds of high speed, medium speed, and low speed
  • the values of the driver state s 2 are -1, -3, -10 , Respectively represent the status of the driver as severe fatigue, mild fatigue, and non-fatigue.
  • the greater the value of s 1 the greater the driving speed, or the greater the value of s 2 , the more tired the driver, and the shorter the working time t 2 of the third cleaning tool.
  • the third preset weight parameter corresponding to the driving state and the fourth preset algorithm Determine the working time of the third cleaning tool.
  • t 2 is the working time of the third cleaning tool
  • t l-max is the maximum continuous working time of the third cleaning tool
  • e is the base of the natural logarithm
  • ⁇ 3 is the third preset weight parameter corresponding to the driving state
  • ⁇ 3 1
  • s 1 represents the driving state
  • the value of s 1 refers to the above.
  • the third preset weight parameter corresponding to the driver state and the fourth preset algorithm Determine the working time of the third cleaning tool.
  • t 2 is the working time of the third cleaning tool
  • t l-max is the maximum continuous working time of the third cleaning tool
  • e is the base of the natural logarithm
  • ⁇ 3 is the third preset weight parameter corresponding to the driver state
  • ⁇ 3 1
  • s 2 represents the driver's state
  • the value of s 2 refers to the above.
  • the working interval of the third cleaning tool can be determined according to the driving state, can also be determined according to the driver's state, or can be determined together with these two factors.
  • the driver state the fourth preset weight parameter corresponding to the driving state
  • the fourth preset weight parameter corresponding to the driver state and the fifth preset algorithm
  • ⁇ t is the working interval of the third cleaning tool
  • ⁇ t max is the maximum working interval of the third cleaning tool
  • e is the base of the natural logarithm
  • ⁇ 4 and ⁇ 4 are respectively the driving state
  • the fourth preset weight parameter corresponding to the driver state, ⁇ 4 + ⁇ 4 1, s 1 represents the driving state, and s 2 represents the driver state.
  • the values of the driving state s 1 are -1, -3, and -10, which respectively represent driving speeds of high speed, medium speed, and low speed
  • the values of the driver state s 2 are -1, -3, -10 , Respectively represent the status of the driver as severe fatigue, mild fatigue, and non-fatigue.
  • the greater the value of s 1 the greater the driving speed, or the greater the value of s 2 , the more tired the driver, and the shorter the working interval ⁇ t of the third cleaning tool.
  • the driver state the fourth preset weight parameter corresponding to the driving state
  • the fourth preset weight parameter corresponding to the driver state and the fifth preset algorithm
  • ⁇ t is the working interval of the third cleaning tool
  • ⁇ t max is the maximum working interval of the third cleaning tool
  • e is the base of natural logarithm
  • ⁇ 4 is the fourth corresponding to the driver state.
  • ⁇ 4 the working interval of the third cleaning tool
  • ⁇ t max the maximum working interval of the third cleaning tool
  • e is the base of the natural logarithm
  • ⁇ 4 is the first corresponding to the driver state.
  • Four preset weight parameters, ⁇ 4 1, s 2 represents the state of the driver, and the value of s 2 refers to the foregoing description.
  • the value of the driving state s 1 and the value of the driver state s 2 may refer to the foregoing.
  • the second cleaning tool is again controlled to clean the first area according to the working time of the second cleaning tool and the working frequency of the second cleaning tool.
  • the specific implementation manner for controlling the second cleaning tool to remove the second-type impurities in the first area may refer to the implementation manner described in (2) of step S405 of the foregoing embodiment, and details are not described herein again.
  • step S701 needs to be re-executed to determine whether the second type of debris on the vehicle window has been cleaned. If the second type of debris in the first area on the car window has been cleaned, the automatic window cleaning process is ended, and the camera in the car is kept on to monitor the car window in real time; if the car window is in the first area The second type of debris is not cleaned (that is, the second type of debris still exists in the first area on the car window), according to the working time of the second cleaning tool, the working frequency of the second cleaning tool, and the working time of the third cleaning tool As well as the working interval of the third cleaning tool, after completing q operations of controlling the second cleaning tool and the third cleaning tool to remove the second type of debris, the automatic window cleaning process is ended, and the camera in the car is turned on to monitor the car windows in real time.
  • q can be determined by the driver himself, or can be preset according to the cleaning effects of the second cleaning tool and the third cleaning tool.
  • the embodiment of the present application after controlling the second cleaning tool to clean the second type of debris in the first area, first determine whether the second type of debris in the first area is cleaned. If it is not cleaned, control the second type of debris. The third cleaning tool and the second cleaning tool clean the second category of debris. Through the cooperation of a variety of cleaning tools, the embodiment of the present application can remove the second-type sundries on the vehicle window as much as possible, thereby achieving a better automatic cleaning effect.
  • FIG. 8 shows the automatic window cleaning device involved in the above embodiment. Schematic diagram of a possible structure.
  • the automatic window cleaning device includes an acquisition unit 801, a determination unit 802, and a control unit 803.
  • the automatic window cleaning device may also include other modules, or the automatic window cleaning device may include fewer modules.
  • the acquiring unit 801 is used to acquire a car window image.
  • the determining unit 802 is configured to determine a dark channel image corresponding to a single frame of the car window image.
  • the determining unit 802 is further configured to determine the presence of the first type of debris on the vehicle window according to the number of pixels in the dark channel image whose gray value exceeds the preset gray threshold and/or the sharpness of the dark channel image.
  • the determining unit 802 is used for the number of pixels in the dark channel image whose gray value does not exceed the preset gray threshold value is less than or equal to the number of pixels in the dark channel image whose gray value exceeds the preset gray threshold value, and/ Or when the sharpness of the dark channel image is less than the preset sharpness, it is determined that there is a first type of debris on the car window.
  • the definition of the dark channel image is the variance of the gray value of the pixels in the dark channel image.
  • And/or the determining unit 802 is further configured to determine that there is a second type of debris on the window of the vehicle according to the RGB values of the pixels in the continuous i frames of the window image.
  • the determining unit 802 is further configured to first establish an image background model according to the RGB values of the pixels in the consecutive i frames of the car window image. According to the RGB values of the pixels in the image background model, the first area is determined, and the second type of debris is determined in the first area.
  • the RGB value of the pixel in the image background model represents the change in the RGB value of the pixel at the same position in the consecutive i frames of the window image, i is an integer greater than 1, and the first area is the RGB value of the consecutive i frames of the window image. The window area corresponding to the changed pixel.
  • the determining unit 802 is further configured to determine the mean image of consecutive i frames of window images according to the mean value of the RGB values of the pixels at the same position in consecutive i frames of window images, and according to the mean image of consecutive i frames of window images The difference between the RGB value of the pixel in the ith frame and the RGB value of the pixel in the i-th frame of the car window image establishes an image background model.
  • the determining unit 802 is also used for the RGB values of the pixels in the image background model and the first preset algorithm to determine the first area.
  • the first preset algorithm is a saliency detection algorithm or a target detection algorithm.
  • the determining unit 802 is further configured to determine a third area corresponding to the first area in the image background model, and determine the RGB value of each pixel in the second area and the RGB value of the pixel at the same position in the third area And accumulate the difference value to get the accumulated value. If the accumulated value exceeds the preset threshold, it is determined that the RGB values of the pixels at the same position in the j second regions have changed.
  • the determining unit 802 first determines a window corresponding to the first region on each frame of the continuous j frames of window images. In the second area, j second areas are obtained. If the RGB values of the pixels at the same position in the j second regions have not changed, it is determined that there is a second type of debris in the first region. Wherein, consecutive j frames of window images are located after consecutive i frames of window images, and j is an integer greater than 1.
  • the control unit 803 is used to control the cleaning tool to remove debris.
  • the sundries include the first type of sundries and/or the second type of sundries.
  • control unit 803 is configured to control the first cleaning tool to be turned on for a preset period of time according to the temperature data inside and outside the vehicle to remove the first type of debris.
  • control unit 803 is configured to control the second cleaning tool to clean the first area according to the working time of the second cleaning tool and the working frequency of the second cleaning tool to remove the second type of debris.
  • the working time of the second cleaning tool is determined according to the driving state and/or the driver state, the first preset weight parameter corresponding to the driving state and/or the driver state, and the maximum duration of continuous operation of the second cleaning tool, and the second cleaning tool
  • the working frequency of the tool is determined according to the driving state and/or the driver state, the second preset weight parameter corresponding to the driving state and/or the driver state, and the maximum frequency of the second cleaning tool.
  • control unit 803 is also used for the first preset weight parameter corresponding to the driving state, the driver state, the first preset weight parameter corresponding to the driving state, the first preset weight parameter corresponding to the driver state, and the maximum continuous operation of the second cleaning tool.
  • control unit 803 is further configured to firstly according to the driving state and/or the driver state, the third preset weight parameter corresponding to the driving state and/or the driver state, and the maximum duration of continuous operation of the third cleaning tool, Determine the working time of the third cleaning tool. Then, according to the driving state and/or the driver state, the fourth preset weight parameter corresponding to the driving state and/or the driver state, and the maximum working interval of the third cleaning tool, the working interval of the third cleaning tool is determined. Finally, the third cleaning tool is controlled to clean the first area according to the working time of the third cleaning tool and the working interval of the third cleaning tool, and the second cleaning tool is controlled to clean the first area according to the working time of the second cleaning tool and the working frequency of the second cleaning tool. Remove the second type of debris.
  • control unit 803 is configured to operate according to the driving state, the driver state, the third preset weight parameter corresponding to the driving state, the third preset weight parameter corresponding to the driver state, the maximum duration of continuous operation of the third cleaning tool, and
  • the fourth preset algorithm determines the working time of the third cleaning tool.
  • the fourth preset algorithm is The control unit 803 is further configured to perform according to the driving state, the driver state, the fourth preset weight parameter corresponding to the driving state, the fourth preset weight parameter corresponding to the driver state, the maximum working interval of the third cleaning tool, and the fifth preset weight parameter.
  • t 2 is the working time of the third cleaning tool
  • t l-max is the maximum continuous working time of the third cleaning tool
  • ⁇ t is the working interval of the third cleaning tool
  • ⁇ t max is the maximum working interval of the third cleaning tool
  • e is the base of the natural logarithm
  • ⁇ 3 and ⁇ 3 are the third preset weight parameters corresponding to the driving state and the driver state
  • ⁇ 3 + ⁇ 3 1
  • ⁇ 4 and ⁇ 4 are the driving state and the driver, respectively
  • the fourth preset weight parameter corresponding to the state, ⁇ 4 + ⁇ 4 1, s 1 represents the driving state
  • s 2 represents the driver state.
  • the present application also provides an automatic cleaning device for vehicle windows, which includes a processor 910 and a memory 920.
  • the processor 910 and the memory 920 are connected (for example, connected to each other through a bus 940).
  • the automatic window cleaning device may further include a transceiver 930, which is connected to the processor 910 and the memory 920, and the transceiver is used to receive/send data.
  • a transceiver 930 which is connected to the processor 910 and the memory 920, and the transceiver is used to receive/send data.
  • the processor 910 can perform operations of any one of the implementations corresponding to FIG. 4, FIG. 6, and FIG. 7 and various feasible implementation manners thereof. For example, it is used to perform operations of the obtaining unit 801, the determining unit 802, and the control unit 803, and/or other operations described in the embodiment of the present application.
  • This application also provides an automatic car window cleaning device, which includes a non-volatile storage medium and a central processing unit.
  • the non-volatile storage medium stores an executable program
  • the central processing unit is connected to the non-volatile storage medium.
  • the executable program is executed to implement the automatic vehicle window cleaning method shown in FIG. 4, FIG. 6 or FIG. 7 in the embodiment of the present application.
  • the present application further provides a computer-readable storage medium.
  • the computer-readable storage medium includes one or more program codes.
  • the one or more programs include instructions.
  • the processor executes the program codes
  • the The automatic vehicle window cleaning device executes the automatic vehicle window cleaning method as shown in FIG. 4, FIG. 6 or FIG. 7.
  • a computer program product in another embodiment of the present application, includes computer-executable instructions, and the computer-executable instructions are stored in a computer-readable storage medium.
  • At least one processor of the automatic car window cleaning device can read the computer-executable instructions from a computer-readable storage medium, and at least one processor executes the computer-executable instructions to make the automatic car window cleaning device implement the instructions shown in FIG. 4, FIG. 6 or FIG. The corresponding steps in the automatic window cleaning method shown.
  • the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium.
  • the technical solution of the present application essentially or the part that contributes to the existing technology or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including Several instructions are used to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in the various embodiments of the present application.
  • the aforementioned storage media include: U disk, mobile hard disk, read-only memory (read-only memory, ROM), random access memory (random access memory, RAM), magnetic disks or optical disks and other media that can store program codes. .

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Mechanical Engineering (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Automation & Control Theory (AREA)
  • Water Supply & Treatment (AREA)
  • Thermal Sciences (AREA)
  • Image Analysis (AREA)
  • Studio Devices (AREA)
  • Stored Programmes (AREA)
  • Traffic Control Systems (AREA)

Abstract

本申请提供了一种车窗自动清洁方法及装置，涉及通信领域，用于根据车窗图像中的像素的取值，检测车窗上的杂物，并控制清洁工具进行自动清洁，减少人工操作。该方法包括：获取车窗图像。确定单帧车窗图像对应的暗通道图像。进而根据暗通道图像中灰度值超过预设灰度阈值的像素的数量和/或暗通道图像的清晰度，确定车窗上存在第一类杂物，和/或根据连续i帧车窗图像中的像素的RGB值，确定车窗上存在第二类杂物。控制清洁工具去除杂物，所述杂物包括第一类杂物和/或第二类杂物。

Description

车窗自动清洁方法及装置
本申请要求于2019年8月23日提交国家知识产权局、申请号为201910786726.5、申请名称为“车窗自动清洁方法及装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及通信技术领域，尤其涉及一种车窗自动清洁方法及装置。
背景技术
车辆在行驶途中，由于车辆内外的温差过大导致车窗上产生水雾甚至霜等，遮挡驾驶员的视线，使得驾驶员需要手动控制除雾系统来进行除雾操作；且由于周围环境的影响，车窗上可能会出现杂物，例如树叶、尘土等，驾驶员也需要手动操作清洁装置，例如雨刮器和清洁液喷洒装置等，去除杂物，给驾驶员的驾驶过程带来不便，降低清洁车窗上的杂物的效率。
发明内容
本申请提供一种车窗自动清洁方法，以实现车窗的自动清洁，提高清洁车窗上的杂物的效率。
为达到上述目的，本申请采用如下技术方案：
第一方面，本申请实施例提供一种车窗自动清洁方法，该方法应用于车辆中，该方法包括：获取车窗图像。确定单帧车窗图像对应的暗通道图像，根据暗通道图像中灰度值超过预设灰度阈值的像素的数量和/或暗通道图像的清晰度，确定车窗上存在第一类杂物；和/或根据连续i帧车窗图像中的像素的RGB值，确定车窗上存在第二类杂物。控制清洁工具去除第一类杂物和/或第二类杂物。
在本申请实施例所描述的车窗自动清洁方法中，利用单帧车窗图像对应的暗通道图像中的像素的灰度值，对车窗上的第一类杂物进行检测，和/或利用连续i帧车窗图像中的像素的RGB值对车窗上的第二类杂物进行检测。若车窗上存在第一类杂物和/或第二类杂物，控制清洁工具进行除杂。首先，在进行杂物检测后进行清洁，可以减少盲目清洁，提高清洁效率，节省清洁资源。另外，通过上述过程，本申请实施例可以实现车窗的自动清洁，减少人工操作，提高车辆驾驶过程中的安全性。
在一种可能的实现方式中，根据连续i帧车窗图像中的像素的RGB值，确定车窗上存在第二类杂物，具体包括：先根据连续i帧车窗图像中像素的RGB值，建立图像背景模型。再根据图像背景模型中的像素的RGB值，确定第一区域，进而确定第一区域内存在第二类杂物。其中，图像背景模型中的像素的RGB值表示连续i帧车窗图像中同一位置的像素的RGB值的变化，i为大于1的整数，第一区域为连续i帧车窗图像中RGB值未发生变化的像素对应的车窗区域。
在本申请实施例所描述的车窗自动清洁方法中，由于在车辆行驶过程中，若车窗上存在落叶等杂物时，车窗上除存在落叶外的位置的画面(像素取值)会随着车辆的行驶发生变化，而车窗上存在落叶的位置的画面(像素取值)不会随着车辆的行驶发生变化或者发生的变化程度较小。因此，根据可以表示连续i帧车窗图像中同一位置 的像素的RGB值的变化的图像背景模型中的像素的RGB值，来确定车窗上存在第二类杂物的第一区域，可以提高确定车窗上存在第二类杂物，以及确定车窗上存在第二类杂物的第一区域的准确性。
在一种可能的实现方式中，在根据图像背景模型中的像素的RGB值，确定第一区域之后，还包括：在连续j帧车窗图像中的每一帧车窗图像上确定与第一区域相对应的一个第二区域，得到j个第二区域。若这j个第二区域内同一位置的像素的RGB值未发生变化，则确定第一区域内存在第二类杂物。其中，连续j帧车窗图像位于连续i帧车窗图像之后，j为大于1的整数。
在本申请实施例所描述的车窗自动清洁方法中，根据连续i帧车窗图像确定车窗上的第一区域之后，进一步根据第一区域以及连续j帧车窗图像中的第二区域，根据第二区域中同一位置的像素的变化来确定第一区域内存在第二类杂物，从而进一步提高检测车窗上存在第二类杂物的准确性。
在一种可能的实现方式中，根据暗通道图像中灰度值超过预设灰度阈值的像素的数量和/或暗通道图像的清晰度，确定车窗上存在第一类杂物，具体包括：若暗通道图像中灰度值未超过预设灰度阈值的像素的数量小于等于暗通道图像中灰度值超过预设灰度阈值的像素的数量，和/或暗通道图像的清晰度小于预设清晰度，则确定车窗上存在第一类杂物。其中，暗通道图像的清晰度为暗通道图像中像素的灰度值的方差。
在一种可能的实现方式中，根据连续i帧车窗图像中像素的RGB值，建立图像背景模型，具体包括：根据连续i帧车窗图像中同一位置的像素的RGB值的均值，确定连续i帧车窗图像的均值图像。然后根据连续i帧车窗图像的均值图像中的像素的RGB值与第i帧车窗图像中的像素的RGB值的差值，建立图像背景模型。
在一种可能的实现方式中，根据图像背景模型中的像素的RGB值，确定第一区域，具体包括：根据图像背景模型中的像素的RGB值以及第一预设算法，确定第一区域。其中，第一预设算法为显著性检测算法或者目标检测算法。
在一种可能的实现方式中，在确定第一区域内存在第二类杂物之前，还包括：先确定图像背景模型中与第一区域相对应的第三区域，然后确定每个第二区域中像素的RGB值与第三区域中同一位置的像素的RGB值的差值并进行累加，得到累加值。若累加值超过预设阈值，则确定j个第二区域内同一位置的像素的RGB值发生变化。
在一种可能的实现方式中，控制清洁工具去除杂物，具体包括：根据车内外温度数据控制第一清洁工具开启预设时长，去除第一类杂物，和/或先根据行驶状态和/或驾驶员状态、行驶状态和/或驾驶员状态对应的第一预设权重参数，以及第二清洁工具连续工作的最大时长，确定第二清洁工具工作时长，再根据行驶状态和/或驾驶员状态、行驶状态和/或驾驶员状态对应的第二预设权重参数，以及第二清洁工具工作的最大频率，确定第二清洁工具工作频率，最后根据第二清洁工具工作时长以及第二清洁工具工作频率控制第二清洁工具清洁第一区域，去除第二类杂物。
在本申请实施例所描述的车窗自动清洁方法中，在不同的行驶状态或者不同的驾驶员状态下，对第二清洁工具的工作时长以及工作频率进行适应性的调整，可以最小程度的减少车窗清洁对驾驶员造成的影响，保证车辆驾驶过程中的安全性。
在一种可能的实现方式中，根据行驶状态和/或驾驶员状态、行驶状态和/或驾驶 员状态对应的第一预设权重参数，以及第二清洁工具连续工作的最大时长，确定第二清洁工具工作时长，具体包括：根据行驶状态、驾驶员状态、行驶状态对应的第一预设权重参数、驾驶员状态对应的第一预设权重参数、第二清洁工具连续工作的最大时长以及第二预设算法，确定第二清洁工具工作时长。其中，第二预设算法为
Figure PCTCN2020102184-appb-000001
t 1为第二清洁工具工作时长，t s-max为第二清洁工具连续工作的最大时长，e为自然对数的底数，α 1和β 1分别为行驶状态和驾驶员状态对应的第一预设权重参数，α 1+β 1＝1，s 1表示行驶状态、s 2表示驾驶员状态。
在一种可能的实现方式中，根据行驶状态和/或驾驶员状态、行驶状态和/或驾驶员状态对应的第二预设权重参数，以及第二清洁工具工作的最大频率，确定第二清洁工具工作频率，具体包括：根据行驶状态、驾驶员状态、行驶状态对应的第二预设权重参数、驾驶员状态对应的第二预设权重参数、第二清洁工具工作的最大频率以及第三预设算法，确定第二清洁工具工作频率。其中，第三预设算法为
Figure PCTCN2020102184-appb-000002
f为第二清洁工具工作频率，f max为第二清洁工具工作的最大频率，e为自然对数的底数，α 2和β 2分别为行驶状态和驾驶员状态对应的第二预设权重参数，α 2+β 2＝1，s 1表示行驶状态、s 2表示驾驶员状态。
在一种可能的实现方式中，在根据第二清洁工具工作时长以及第二清洁工具工作频率控制第二清洁工具清洁第一区域之后，还包括：先根据行驶状态和/或驾驶员状态、行驶状态和/或驾驶员状态对应的第三预设权重参数，以及第三清洁工具连续工作的最大时长，确定第三清洁工具工作时长。再根据行驶状态和/或驾驶员状态、行驶状态和驾驶员状态对应的第四预设权重参数，以及第三清洁工具工作的最大间隔，确定第三清洁工具工作间隔。最后根据第三清洁工具工作时长以及第三清洁工具工作间隔控制第三清洁工具清洁第一区域，并根据第二清洁工具工作时长以及第二清洁工具工作频率控制第二清洁工具清洁第一区域，去除第二类杂物。
在本申请实施例所描述的车窗自动清洁方法中，在不同的行驶状态或者不同的驾驶员状态下，对第二清洁工具的工作时长以及工作频率进行适应性的调整，在可以最小程度的减少车窗清洁对驾驶员造成的影响，保证车辆驾驶过程中的安全性。利用多种清洁工具对车窗进行自动清洁可以最大程度的提高清洁效率，从而进一步提高车辆驾驶过程中的安全性。
在一种可能的实现方式中，根据行驶状态和/或驾驶员状态、行驶状态和/或驾驶员状态对应的第三预设权重参数，以及第三清洁工具连续工作的最大时长，确定第三清洁工具工作时长，具体包括：根据行驶状态、驾驶员状态、行驶状态对应的第三预设权重参数、驾驶员状态对应的第三预设权重参数、第三清洁工具连续工作的最大时长以及第四预设算法，确定第三清洁工具工作时长。其中，第四预设算法为
Figure PCTCN2020102184-appb-000003
t 2为第三清洁工具工作时长，t l-max为第三清洁工具连续工作的最大时长，e为自然对数的底数，α 3和β 3分别为所述行驶状态和所述驾驶员状态对应的第三预设权重参数，α 3+β 3＝1，s 1表示行驶状态、s 2表示驾驶员状态。
在一种可能的实现方式中，根据行驶状态和/或驾驶员状态、行驶状态和驾驶员状态对应的第四预设权重参数，以及第三清洁工具工作的最大间隔，确定第三清洁工具工作间隔，具体包括：根据行驶状态、驾驶员状态、行驶状态对应的第四预设权重参 数、驾驶员状态对应的第四预设权重参数、第三清洁工具工作的最大间隔以及第五预设算法，确定第三清洁工具工作间隔。其中，第五预设算法为
Figure PCTCN2020102184-appb-000004
△t为第三清洁工具工作间隔，△t max为第三清洁工具工作的最大间隔，e为自然对数的底数，α 4和β 4分别为行驶状态和驾驶员状态对应的第四预设权重参数，α 4+β 4＝1，s 1表示行驶状态、s 2表示驾驶员状态。
第二方面，本申请实施例提供一种车窗自动清洁装置，该装置应用于车辆中该方法包括获取单元、确定单元以及控制单元。获取单元，用于获取车窗图像。确定单元，用于确定单帧车窗图像对应的暗通道图像。确定单元，还用于根据暗通道图像中灰度值超过预设灰度阈值的像素的数量和/或暗通道图像的清晰度，确定车窗上存在第一类杂物；和/或确定单元，还用于根据连续i帧车窗图像中的像素的RGB值，确定车窗上存在第二类杂物。控制单元，用于控制清洁工具去除第一类杂物和/或第二类杂物。
在一种可能的实现方式中，确定单元，具体用于先根据连续i帧车窗图像中像素的RGB值，建立图像背景模型。再根据图像背景模型中的像素的RGB值，确定第一区域，进而确定第一区域内存在第二类杂物。其中，第一区域为连续i帧车窗图像中RGB值未发生变化的像素对应的车窗区域，图像背景模型中的像素的RGB值表示连续i帧车窗图像中同一位置的像素的RGB值的变化，i为大于1的整数。
在一种可能的实现方式中，确定单元，在根据图像背景模型中的像素的RGB值，确定第一区域之后，还用于在连续j帧车窗图像中的每一帧车窗图像上确定与第一区域相对应的一个第二区域，得到j个第二区域。若这j个第二区域内同一位置的像素的RGB值未发生变化，则确定第一区域内存在第二类杂物。其中，连续j帧车窗图像位于连续i帧车窗图像之后，j为大于1的整数。
在一种可能的实现方式中，确定单元，具体用于若暗通道图像中灰度值未超过预设灰度阈值的像素的数量小于等于暗通道图像中灰度值超过预设灰度阈值的像素的数量，和/或暗通道图像的清晰度小于预设清晰度，则确定车窗上存在第一类杂物。其中，暗通道图像的清晰度为暗通道图像中像素的灰度值的方差。
在一种可能的实现方式中，确定单元，具体用于根据连续i帧车窗图像中同一位置的像素的RGB值的均值，确定连续i帧车窗图像的均值图像。然后根据连续i帧车窗图像的均值图像中的像素的RGB值与第i帧车窗图像中的像素的RGB值的差值，建立图像背景模型。
在一种可能的实现方式中，确定单元，具体用于根据图像背景模型中的像素的RGB值以及第一预设算法，确定第一区域。其中，第一预设算法为显著性检测算法或者目标检测算法。
在一种可能的实现方式中，确定单元，还用于先确定图像背景模型中与第一区域相对应的第三区域，然后确定每个第二区域中像素的RGB值与第三区域中同一位置的像素的RGB值的差值并进行累加，得到累加值。若累加值超过预设阈值，则确定j个第二区域内同一位置的像素的RGB值发生变化。
在一种可能的实现方式中，控制单元，具体用于根据车内外温度数据控制第一清洁工具开启预设时长，去除第一类杂物，和/或先根据行驶状态和/或驾驶员状态、行驶状态和/或驾驶员状态对应的第一预设权重参数，以及第二清洁工具连续工作的最大 时长，确定第二清洁工具工作时长，再根据行驶状态和/或驾驶员状态、行驶状态和/或驾驶员状态对应的第二预设权重参数，以及第二清洁工具工作的最大频率，确定第二清洁工具工作频率，最后根据第二清洁工具工作时长以及第二清洁工具工作频率控制第二清洁工具清洁第一区域，去除第二类杂物。
在一种可能的实现方式中，控制单元，具体用于根据行驶状态和/或驾驶员状态、行驶状态和/或驾驶员状态对应的第一预设权重参数，以及第二清洁工具连续工作的最大时长，确定第二清洁工具工作时长，具体包括：根据行驶状态、驾驶员状态、行驶状态对应的第一预设权重参数、驾驶员状态对应的第一预设权重参数、第二清洁工具连续工作的最大时长以及第二预设算法，确定第二清洁工具工作时长。其中，第二预设算法为
Figure PCTCN2020102184-appb-000005
t 1为第二清洁工具工作时长，t s-max为第二清洁工具连续工作的最大时长，e为自然对数的底数，α 1和β 1分别为行驶状态和驾驶员状态对应的第一预设权重参数，α 1+β 1＝1，s 1表示行驶状态、s 2表示驾驶员状态。
在一种可能的实现方式中，控制单元，具体用于根据行驶状态和/或驾驶员状态、行驶状态和/或驾驶员状态对应的第二预设权重参数，以及第二清洁工具工作的最大频率，确定第二清洁工具工作频率，具体包括：根据行驶状态、驾驶员状态、行驶状态对应的第二预设权重参数、驾驶员状态对应的第二预设权重参数、第二清洁工具工作的最大频率以及第三预设算法，确定第二清洁工具工作频率。其中，第三预设算法为
Figure PCTCN2020102184-appb-000006
f为第二清洁工具工作频率，f max为第二清洁工具工作的最大频率，e为自然对数的底数，α 2和β 2分别为行驶状态和驾驶员状态对应的第二预设权重参数，α 2+β 2＝1，s 1表示行驶状态、s 2表示驾驶员状态。
在一种可能的实现方式中，控制单元，具体用于在根据第二清洁工具工作时长以及第二清洁工具工作频率控制第二清洁工具清洁第一区域之后，还包括：先根据行驶状态和/或驾驶员状态、行驶状态和/或驾驶员状态对应的第三预设权重参数，以及第三清洁工具连续工作的最大时长，确定第三清洁工具工作时长。再根据行驶状态和/或驾驶员状态、行驶状态和驾驶员状态对应的第四预设权重参数，以及第三清洁工具工作的最大间隔，确定第三清洁工具工作间隔。最后根据第三清洁工具工作时长以及第三清洁工具工作间隔控制第三清洁工具清洁第一区域，并根据第二清洁工具工作时长以及第二清洁工具工作频率控制第二清洁工具清洁第一区域，去除第二类杂物。
在一种可能的实现方式中，控制单元，具体用于根据行驶状态和/或驾驶员状态、行驶状态和/或驾驶员状态对应的第三预设权重参数，以及第三清洁工具连续工作的最大时长，确定第三清洁工具工作时长，具体包括：根据行驶状态、驾驶员状态、行驶状态对应的第三预设权重参数、驾驶员状态对应的第三预设权重参数、第三清洁工具连续工作的最大时长以及第四预设算法，确定第三清洁工具工作时长。其中，第四预设算法为
Figure PCTCN2020102184-appb-000007
t 2为第三清洁工具工作时长，t l-max为第三清洁工具连续工作的最大时长，e为自然对数的底数，α 3和β 3分别为所述行驶状态和所述驾驶员状态对应的第三预设权重参数，α 3+β 3＝1，s 1表示行驶状态、s 2表示驾驶员状态。
在一种可能的实现方式中，控制单元，具体用于根据行驶状态和/或驾驶员状态、行驶状态和驾驶员状态对应的第四预设权重参数，以及第三清洁工具工作的最大间隔，确定第三清洁工具工作间隔，具体包括：根据行驶状态、驾驶员状态、行驶状态 对应的第四预设权重参数、驾驶员状态对应的第四预设权重参数、第三清洁工具工作的最大间隔以及第五预设算法，确定第三清洁工具工作间隔。其中，第五预设算法为
Figure PCTCN2020102184-appb-000008
△t为第三清洁工具工作间隔，△t max为第三清洁工具工作的最大间隔，e为自然对数的底数，α 4和β 4分别为行驶状态和驾驶员状态对应的第四预设权重参数，α 4+β 4＝1，s 1表示行驶状态、s 2表示驾驶员状态。
第三方面，提供一种车窗自动清洁装置，包括：处理器和存储器；该存储器用于存储计算机执行指令，当该车窗自动清洁装置运行时，该处理器执行该存储器存储的该计算机执行指令，以使该车窗自动清洁装置执行如上述第一方面以及第一方面中任一项的车窗自动清洁方法。
第四方面，本申请实施例中还提供一种计算机可读存储介质，包括指令，当其在计算机上运行时，使得计算机执行如上述第一方面以及第一方面中任一项的车窗自动清洁方法。
第五方面，本申请实施例中还提供一种计算机程序产品，包括指令，当其在计算机上运行时，使得计算机执行如上述第一方面以及第一方面中任一项的车窗自动清洁方法。
附图说明
图1为本申请实施例提供的一种车辆的结构示意图一；
图2为本申请实施例提供的一种车辆的结构示意图二；
图3为本申请实施例提供的一种计算机系统的结构示意图；
图4为本申请实施例提供的车窗自动清洁方法流程示意图一；
图5为本申请实施例提供的一种车内摄像头安装位置示意图；
图6为本申请实施例提供的车窗自动清洁方法流程示意图二；
图7为本申请实施例提供的车窗自动清洁方法流程示意图三；
图8为本申请实施例提供的车窗自动清洁装置的结构示意图一；
图9为本申请实施例提供的车窗自动清洁装置的结构示意图二。
具体实施方式
本申请实施例提供的车窗自动清洁方法应用在车辆中，或者应用于具有车窗清洁功能的其他设备(比如云端服务器)中。车辆可通过其包含的组件(包括硬件和软件)实施本申请实施例提供的车窗自动清洁方法，检测并自动清洁车窗上的杂物。或者，通过其他设备(比如服务器、手机终端等)实施本申请实施例的车窗自动清洁方法，在检测到车窗上存在杂物后进行自动清洁，以减少人工操作，提高车辆驾驶的安全性。
图1是本申请实施例提供的车辆100的功能框图。在一个实施例中，车辆100可以根据车内摄像头收集到的车窗图像确定车窗上是否存在杂物，进而控制车辆内的清洁工具进行自动车窗清洁，去除车窗上的杂物。
车辆100可包括各种子系统，例如行进系统110、传感器系统120、控制系统130、无线通信系统140、电源150、计算机系统160以及用户接口170。可选地，车辆100可包括更多或更少的子系统，并且每个子系统可包括多个元件。另外，车辆100的每个子系统和元件可以通过有线或者无线互连。
行进系统110可包括为车辆100提供动力的组件，例如引擎、传动装置等。
传感器系统120可包括感测关于车辆100周边的环境的信息的若干个传感器。例如，传感器系统120可包括定位系统121(定位系统可以是GPS系统，也可以是北斗系统或者其他定位系统)、惯性测量单元(inertial measurement unit，IMU)122、雷达传感器123、激光雷达124、视觉传感器125、超声波传感器126(图中未示出)以及声纳传感器127(图中未示出)中的至少一个。可选地，传感器系统120还可包括被监视车辆100的内部系统的传感器(例如，车内摄像头、车内空气质量监测器、燃油量表、机油温度表等)。来自这些传感器中的一个或多个的传感器数据可用于检测对象及其相应特性(位置、形状、方向、速度等)。这种检测和识别是保证车辆100的安全操作的关键功能。
定位系统121可用于估计车辆100的地理位置。IMU 122用于基于惯性加速度来感测车辆100的位置和朝向变化。在一个实施例中，IMU 122可以是加速度计和陀螺仪的组合。
雷达传感器123可利用电磁波信号来感测车辆100的周边环境内的物体。在一些实施例中，除了感测物体的位置以外，雷达传感器123还可用于感测物体的径向速度和/或该物体的雷达散射截面积RCS。
激光雷达124可利用激光来感测车辆100所位于的环境中的物体。在一些实施例中，激光雷达124可包括一个或多个激光源、激光扫描器以及一个或多个检测器，以及其他系统组件。
视觉传感器125可用于捕捉车辆100的周边环境的多个图像、车内环境的多个图像，其中，车内环境的多个图像中包括车窗图像。视觉传感器125可以是静态相机或视频相机。
控制系统130可控制车辆100及其组件的操作。控制系统130可包括各种元件，例如计算机视觉系统131、路线控制系统132、障碍规避系统133以及清洁系统134等系统中的至少一个。
计算机视觉系统131可以操作来处理和分析由视觉传感器125捕捉的图像以及由雷达传感器123得到的测量数据，以便识别车辆100周边环境中的物体和/或特征以及车辆内部，例如车窗上的杂物。车辆100周边环境中的物体和/或特征可包括交通信号、道路边界和障碍物。计算机视觉系统131可使用物体识别算法、运动中恢复结构(structure from motion，SFM)算法、视频跟踪和其他计算机视觉技术。在一些实施例中，计算机视觉系统131可以用于为环境绘制地图、跟踪物体、估计物体的速度等等。
路线控制系统132用于确定车辆100的行驶路线。在一些实施例中，路线控制系统132可结合来自雷达传感器123、定位系统121和一个或多个预定地图的数据以为车辆100确定行驶路线。
障碍规避系统133用于识别、评估和避免或者以其他方式越过车辆100的环境中的潜在障碍物。
清洁系统134用于对车窗上的杂物进行清理，该清洁系统134中包括空调、清洁液喷洒器、雨刷等清洁工具。在本申请实施例中，清洁系统134可以结合来自视觉传感器125的车窗图像信息，对车窗上存在的杂物进行清理，以提高驾驶员在驾驶车辆过程中的安全性。
当然，在一个实例中，控制系统130可以增加或替换地包括除了所示出和描述的那些以外的组件。或者也可以减少一部分上述示出的组件。
车辆100可利用无线通信系统140获取所需信息，其中，无线通信系统140可以直接地或者经由通信网络来与一个或多个设备无线通信。例如，无线通信系统140可使用3G蜂窝通信，例如CDMA、EVD0、GSM/GPRS，或者4G蜂窝通信，例如LTE。或者5G蜂窝通信。无线通信系统140可利用WiFi与无线局域网(wireless local area network，WLAN)通信。在一些实施例中，无线通信系统140可利用红外链路、蓝牙或ZigBee与设备直接通信。其他无线协议，例如各种车辆通信系统，例如，无线通信系统140可包括一个或多个专用短程通信(dedicated short range communications，DSRC)设备。
车辆100的部分或所有功能受计算机系统160控制。计算机系统160可包括至少一个处理器161，处理器161执行存储在例如数据存储装置162这样的非暂态计算机可读介质中的指令1621。计算机系统160还可以是采用分布式方式控制车辆100的个体组件或子系统的多个计算设备。
处理器161可以是任何常规的处理器，诸如商业可获得的中央处理单元(central processing unit，CPU)。替选地，该处理器可以是诸如专用集成电路(application specific integrated circuit，ASIC)或其它基于硬件的处理器的专用设备。尽管图1功能性地图示了处理器、存储器、和在相同物理外壳中的其它元件，但是本领域的普通技术人员应该理解该处理器、计算机系统、或存储器实际上可以包括可以存储在相同的物理外壳内的多个处理器、计算机系统、或存储器，或者包括可以不存储在相同的物理外壳内的多个处理器、计算机系统、或存储器。例如，存储器可以是硬盘驱动器，或位于不同于物理外壳内的其它存储介质。因此，对处理器或计算机系统的引用将被理解为包括对可以并行操作的处理器或计算机系统或存储器的集合的引用，或者可以不并行操作的处理器或计算机系统或存储器的集合的引用。不同于使用单一的处理器来执行此处所描述的步骤，诸如转向组件和减速组件的一些组件每个都可以具有其自己的处理器，所述处理器只执行与特定于组件的功能相关的计算。
在此处所描述的各个方面中，处理器可以位于远离该车辆并且与该车辆进行无线通信。在其它方面中，此处所描述的过程中的一些在布置于车辆内的处理器上执行而其它则由远程处理器执行，包括采取执行单一操纵的必要步骤。
可选地，上述组件只是一个示例，实际应用中，上述各个模块中的组件有可能根据实际需要增添或者删除，图1不应理解为对本申请实施例的限制。
在道路上行进的汽车，如上面的车辆100，可以识别车窗上的杂物，并确定相应的清洁策略，使得车辆可以自动的进行车窗清洁。在一些示例中，可以独立地考虑每个识别的杂物，并且基于杂物各自的特性，诸如它的形状、面积等以及本车的行驶状态(例如速度)、驾驶员状态(例如驾驶员是否疲劳)，可以用来确定驾驶中汽车的车窗自动清洁策略。
可选地，车辆100或者与车辆100相关联的计算设备(如图1的计算机系统160、计算机视觉系统131、数据存储装置162)可以基于所获取的车窗图像来检测和识别车窗上的杂物。车辆100能够基于预测的车窗上的杂物以及车辆的行驶状态和驾驶员 状态来调整它的清洁策略。换句话说，车辆能够基于所预测的杂物来确定车辆中的清洁工具将需要工作的时间以及频率。在这个过程中，也可以考虑其它因素来确定车辆100的车窗自动清洁策略，诸如，车辆100在行驶过程中周围车辆的状态、天气状况等等。
除了提供用于识别车窗上的杂物，以确定行驶汽车的车窗自动清洁策略之外，计算设备还可以提供车辆100指示自动清洁车窗杂物的指令，以使得驾驶过程中的汽车在保证安全的情况下，自动清洁其车窗上的杂物(例如，空调吹热风/冷风、喷洒清洁液、挥动雨刷)，使车窗保持洁净。
上述车辆100可以为轿车、卡车、摩托车、公共汽车、船、飞机、直升飞机、娱乐车、施工设备、电车、高尔夫球车、和火车等，本申请实施例不做特别的限定。
参见图2，示例性的，车辆中可以包括以下模块：
环境感知模块201，用于获取车内摄像头拍摄到的车窗图像，结合车窗图像确定车窗上是否存在杂物，包括第一类杂物和第二类杂物，并确定第二类杂物的位置。另外，环境感知模块201中至少包括温度传感器以及摄像头等检测装置。通过车内的摄像头实时获取车窗图像以及车窗视频等信息，通过温度传感器实时获取车内外温度数据等信息，并将所获取到的信息传递给中央处理模块202，以便于中央处理模块202生成相应的车窗自动清洁策略。
中央处理模块202(例如车载计算机)，用于从环境感知模块201接收车窗图像、车窗视频以及车内外温度数据等信息，中央处理模块202结合其内部保存的当前车辆的行驶状态(例如速度)以及驾驶员状态(例如驾驶员是否疲劳、驾驶员是否分心)，对从环境感知模块201接收到的车窗图像以及车窗视频进行检测和分析，确定车窗上存在第一类杂物和/或第二类杂物，并生成相应的清洁决策(例如雨刷的工作时长和工作频率等)，输出清洁决策所对应的动作指令，并向动作执行模块203发送该动作指令，以指示动作执行模块203按照动作指令对车窗进行自动清洁。
动作执行模块203，用于从中央处理模块202接收动作指令，并按照动作指令完成自动车窗清洁的操作。其中，动作执行模块中至少包括车辆内的清洁工具，例如雨刷、空调、清洁液喷洒器等。
车载通信模块204(图2中并未示出)：用于自车和其他车之间的信息交互。
存储组件205(图2中并未示出)，用于存储上述各个模块的可执行代码。运行这些可执行代码可实现本申请实施例的部分或全部方法流程。
在本申请实施例的一种可能的实现方式中，如图3所示，图1所示的计算机系统160包括处理器301，处理器301和系统总线302耦合。处理器301可以是一个或者多个处理器，其中每个处理器都可以包括一个或多个处理器核。显示适配器(video adapter)303，显示适配器303可以驱动显示器309，显示器309和系统总线302耦合。系统总线302通过总线桥304和输入输出(I/O)总线(BUS)305耦合。I/O接口306和I/O总线305耦合。I/O接口306和多种I/O设备进行通信，比如输入设备307(如：键盘，鼠标，触摸屏等)，多媒体盘(media tray)308，(例如，CD-ROM，多媒体接口等)。收发器315(可以发送和/或接收无线电通信信号)，摄像头310(可以捕捉静态和动态数字视频图像)和外部通用串行总线(universal serial bus，USB)接口311。其中，可选地，和I/O接口306相 连接的接口可以是USB接口。
其中，处理器301可以是任何传统处理器，包括精简指令集计算(reduced instruction set computer，RISC)处理器、复杂指令集计算(complex instruction set computer，CISC)处理器或上述的组合。可选地，处理器可以是诸如专用集成电路(ASIC)的专用装置。可选地，处理器301可以是神经网络处理器或者是神经网络处理器和上述传统处理器的组合。
可选地，在本文所述的各种实施例中，计算机系统160可位于远离车辆的地方，并且可与车辆100无线通信。在其它方面，本文所述的一些过程可设置在车辆内的处理器上执行，其它一些过程由远程处理器执行，包括采取执行单个操纵所需的动作。
计算机系统160可以通过网络接口312和软件部署服务器(deploying server)313通信。网络接口312是硬件网络接口，比如，网卡。网络(network)314可以是外部网络，比如因特网，也可以是内部网络，比如以太网或者虚拟私人网络(VPN)。可选地，网络314还可以为无线网络，比如WiFi网络，蜂窝网络等。
在本申请的另一些实施例中，本申请实施例的车窗自动清洁方法还可以由芯片系统执行。本申请实施例提供了一种芯片系统。由主CPU(Host CPU)和神经网络处理器(neural processing unit，NPU)共同配合，可实现图1中车辆100所需功能的相应算法，也可实现图2所示车辆所需功能的相应算法，也可以实现图3所示计算机系统160所需功能的相应算法。
在本申请的另一些实施例中，计算机系统160还可以从其它计算机系统接收信息或转移信息到其它计算机系统。或者，从车辆100的传感器系统120收集的传感器数据可以被转移到另一个计算机，由另一计算机对此数据进行处理。来自计算机系统160的数据可以经由网络被传送到云侧的计算机系统用于进一步的处理。网络以及中间节点可以包括各种配置和协议，包括因特网、万维网、内联网、虚拟专用网络、广域网、局域网、使用一个或多个公司的专有通信协议的专用网络、以太网、WiFi和HTTP、以及前述的各种组合。这种通信可以由能够传送数据到其它计算机和从其它计算机传送数据的任何设备执行，诸如调制解调器和无线接口。
本申请实施例提供的车窗自动清洁方法应用在车窗清洁的场景中，可以由车辆中的控制芯片、处理器等执行，例如图1中的处理器161或图3中的处理器301，另外，本申请实施例提供的车窗自动清洁方法也可以由具有车窗清洁功能的其他设备，例如云端服务器来执行，下面结合附图详细描述本申请实施例的车窗自动清洁方法。
为了实现车窗的自动清洁，本申请提供一种车窗自动清洁方法，该方法包括如下步骤S401-S405，下面结合图4和图5，对本申请的实施例进行描述：
S401、获取车窗图像。
其中，本申请实施例所指的车窗包括前挡风玻璃、后挡风玻璃、侧面车窗玻璃。相应的，车窗图像中是指车辆的前挡风玻璃图像和/或车辆的后挡风玻璃图像。可选的，车窗图像还可以为车辆侧面车窗玻璃图像。车窗图像中不包括车体其他部分。
可选的，利用车内的摄像头拍摄得到车窗图像。车内摄像头的数量可以为一个，也可以为多个。摄像头需要设置在能够获取到车窗图像的位置。
示例性的，如图5所示，区域501表示车辆的前挡风玻璃，区域502表示车辆的后挡风玻璃，区域503和区域504表示车辆侧面的车窗玻璃。车内摄像头可以设置在 车辆的后挡风玻璃与车辆内顶部相连接的位置，如506和508所示的位置，摄像头位于该位置能够获取到车辆的后挡风玻璃图像。车内摄像头还可以设置在车辆侧面的车窗玻璃与车辆内顶部相连接的位置，如505和509所示的位置，摄像头位于该位置能够获取到车辆的前挡风玻璃图像、车辆的后挡风玻璃图像以及车辆侧面的车窗玻璃图像。车内摄像头还可以设置在车辆内顶部上，如507所示的位置，摄像头位于该位置能够获取到车辆的前挡风玻璃图像、车辆的后挡风玻璃图像或者车辆侧面的车窗玻璃图像。另外，需要说明的是，车内摄像头的位置可自行根据需求确定，并不局限于图5所示的位置。示例性的，摄像头也可以放置于车辆前挡风玻璃与车辆内顶相连接的位置，摄像头位于该位置能够获取到车辆的前挡风玻璃图像。
可选的，为了能够在检测到前挡风玻璃上存在杂物时，进行自动清洁，以减少前挡风玻璃上存在的杂物对驾驶员视线的遮挡，提高车辆行驶过程中的安全性，车内摄像头的位置需至少设置为车内可以拍摄到车辆前挡风玻璃(尤其是车辆驾驶员侧的前挡风玻璃)的位置。这样，在车辆行驶过程中，摄像头可以实时拍摄得到车辆的前挡风玻璃图像，从而实时检测前挡风玻璃上是否存在杂物。
本步骤中，获取车窗图像时，可以单次拍摄得到单帧车窗图像，也可以多次连续拍摄，得到多帧车窗图像。参见下述S402和S403，本申请实施例可以根据单帧车窗图像检测车窗上是否存在第一类杂物，如雾、霜等。参见下述S404，本申请实施例可以根据多帧车窗图像检测车窗上是否存在第二类杂物，如灰尘、落叶等杂质。
S402、确定单帧车窗图像对应的暗通道图像。
其中，单帧车窗图像为上述步骤S401中摄像头单次拍摄得到的，也可以是上述步骤S401中摄像头多次连续拍摄的得到的多帧车窗图像中的最后一帧，也可以是上述步骤S401中摄像头多次连续拍摄的得到的多帧车窗图像中的任意一帧。暗通道图像为单帧车窗图像的灰度值图像，相对于单帧车窗图像来说，其灰度值图像能够更为清楚地反映车窗上是否有雾。因此，本申请实施例中利用单帧车窗的暗通道图像来检测车窗上是否有雾。暗通道图像的每个像素与单帧车窗图像的每个像素之间一一对应，暗通道图像的每个像素的取值根据对应的同一位置的单帧车窗图像中的像素的取值计算得到。
以车窗图像中的任意像素x为例，根据单帧车窗图像中像素x的RGB值以及暗通道模型
Figure PCTCN2020102184-appb-000009
进行计算，确定该像素x的灰度值，像素的灰度值的取值范围为0～255。其中，I表示单帧车窗图像，c表示像素的颜色通道，包括r通道、g通道、以及b通道，即红色通道、绿色通道和蓝色通道，I c(x)表示像素x在r通道、g通道、b通道三个颜色通道上的取值，w为单帧车窗图像中以像素x为中心的大小为a*a的像素区域内的像素集合，a为大于0的整数，a的值可以根据实际需要确定，较为常见的a的值为1、3、5、7等。J dark表示像素x的灰度值，J dark的取值为w中像素在颜色通道(包括r、g、b三个颜色通道)上的最小值。
例如，单帧车窗图像中包括有5*5个像素，像素x为单帧车窗图像中第3行第3列的像素。w为单帧车窗图像中以像素x为中心的大小为3*3的像素区域内的像素集合，则该像素集合w包括单帧车窗图像中第2行第2列、第2行第3列、第2行第4列、第3行第2列、第3行第3列、第3行第4列、第4行第2列、第4行第3列、 以及第4行第4列的像素。这些像素的RGB值分别为(255，146，85)、(254，150，90)、(240，165，100)、(244，163，98)、(240，159，95)、(248，160，99)、(249，159，89)、(239，149，85)、以及(245，157，99)。根据上述暗通道模型，确定w中的3*3个像素在r通道、g通道以及b通道三个颜色通道上的最小取值为85，则待处理像素x的灰度值为85，也即像素x在暗通道图像中对应的像素的RGB值为(85，85，85)。
对于单帧车窗图像中的边缘像素x，以边缘像素x为中心的大小为a*a的像素区域内的像素集合w中的像素数量小于等于a*a个。
例如，单帧车窗图像中包括有5*5个像素，像素x为单帧车窗图像中第5行第5列的像素。w为单帧车窗图像中以像素x为中心的大小为3*3的像素区域内的像素集合，则该像素集合w包括单帧车窗图像中第4行第4列、第4行第5列、第5行第4列、以及第5行第5列的像素。这些像素的RGB值分别为(245，157，99)、(255，159，90)、(244，163，95)、以及(254，165，89)。根据上述暗通道模型，确定w中的4个像素在r通道、g通道以及b通道三个颜色通道上的最小取值为89，则像素x的灰度值为89，也即像素x在暗通道图像中对应的像素的RGB值为(89,89,89)。
S403、根据暗通道图像中灰度值超过预设灰度阈值的像素的数量和/或暗通道图像的清晰度，确定车窗上存在第一类杂物。
示例性的，第一类杂物为雾或者霜等杂物。
车窗上存在第一类杂物可以根据暗通道图像中的灰度值超过预设灰度阈值的像素的数量确定，也可以根据暗通道图像的清晰度确定，也可以结合这两个因素一起确定。因此，确定车窗上存在第一类杂物的方法有三种，如下(1)-(3)所示：
(1)根据暗通道图像中灰度值超过预设灰度阈值的像素的数量，确定车窗上存在第一类杂物。若暗通道图像中灰度值未超过预设灰度阈值的像素的数量小于等于暗通道图像中灰度值超过预设灰度阈值的像素的数量，则确定车窗上存在第一类杂物。
示例性的，单帧车窗图像中包含有10*10个像素，则其对应的暗通道图像为该单帧车窗图像的灰度值图像，该暗通道图像中包含有10*10个像素，与单帧车窗图像中的像素相对应，像素的灰度值的取值范围为0～255。若预设灰度阈值为100，该暗通道图像中灰度值未超过预设灰度阈值的像素的数量为30，灰度值超过预设灰度阈值的像素的数量为70，30<70，因此车窗上存在第一类杂物；若预设灰度阈值为100，该暗通道图像中灰度值未超过预设灰度阈值的像素的数量为50，灰度值超过预设灰度阈值的像素的数量为50，因此，车窗上存在第一类杂物；若预设灰度阈值为100，该暗通道图像中灰度值未超过预设灰度阈值的像素的数量为70，灰度值超过预设灰度阈值的像素的数量为30，70>30，因此，车窗上不存在第一类杂物。
示例性的，单帧车窗图像中包含有10*10个像素，则其对应的暗通道图像为该单帧车窗图像的灰度值图像，该暗通道图像中包含有10*10个像素的灰度图像，与单帧车窗图像中的像素相对应，像素的灰度值取值范围为0～255。以像素的灰度值为横坐标，以每个同一灰度值的像素的数量为纵坐标，建立该暗通道图像的灰度值直方图。统计像素的灰度值为灰度值取值范围的前三分之一(即灰度值为0～85)的像素的数量sum1，以及像素的灰度值为灰度值取值范围的后三分之二(即灰度值为86～255)的 像素的数量sum2。若sum1<＝sum2，则车窗上存在第一类杂物；若sum1>sum2，则车窗上不存在第一类杂物。需要说明的是，本申请实施例在采用三分之一时有较好的效果，但并不限定于三分之一，可以根据实际情况来确定。
(2)根据暗通道图像的清晰度，确定车窗上存在第一类杂物。若暗通道图像的清晰度小于预设清晰度，则确定车窗上存在第一类杂物。
可选的，暗通道图像的清晰度为暗通道图像中的像素的灰度值的方差。
示例性的，单帧车窗图像中包含有2*2个像素，则其对应的暗通道图像为包含有2*2个像素(这2*2个像素分别为A、B、C、D)的灰度图像，像素的灰度值的取值范围为0～255。在暗通道图像中，像素A的灰度值为7，像素B的灰度值为1，像素C的灰度值为3，像素D的灰度值为5，像素A、B、C、D的灰度值的平均值为5，则该暗通道图像的清晰度(即该暗通道图像中的像素的灰度值的方差)为[(7-4) 2+(1-4) 2+(3-4) 2+(5-4) 2]/4＝5。若预设清晰度为6，则车窗上存在第一类杂物；若预设清晰度为5，则车窗上不存在第一类杂物；若预设清晰度为4，则车窗上不存在第一类杂物。
可选的，暗通道图像的清晰度根据暗通道图像中的像素的信息熵确定。其中，暗通道图像的信息熵通过
Figure PCTCN2020102184-appb-000010
来确定，H为暗通道图像的信息熵，d表示像素的灰度值，n表示像素的灰度值的最大取值，p d表示灰度值为d的像素在暗通道图像中出现的概率。暗通道图像的信息熵可以表示图像中的灰度分布情况，暗通道图像的信息熵越大，则该暗通道图像中的像素的灰度值变化范围越大，图像的清晰度越高，暗通道图像的信息熵越小，则该暗通道图像中的像素的灰度值变化范围越小，图像的清晰度越低。若暗通道图像的信息熵小于预设信息熵阈值，则车窗上存在第一类杂物，若暗通道图像的信息熵不小于预设信息熵阈值，则车窗上不存在第一类杂物。
可选的，暗通道图像的清晰度为利用HSV(hue，saturation，value)颜色模型进行计算得到的暗通道图像的饱和度。暗通道图像的饱和度越高，则该暗通道图像的清晰度越高，暗通道图像的饱和度越低，则该暗通道图像的清晰度越低。若暗通道图像的饱和度小于预设饱和度阈值，则车窗上存在第一类杂物，若暗通道图像的饱和度不小于预设饱和度阈值，则车窗上不存在第一类杂物。
需要说明的是，用于确定或表示图像的清晰度的度量方式并不局限于本申请实施例中所给出的信息熵与饱和度，HSV颜色模型中的色相(hue)和明度(value)也可以用于确定或表示图像的清晰度，图像的清晰度的具体度量方式可以根据实际应用来确定。
(3)根据暗通道图像中灰度值超过预设灰度阈值的像素的数量和暗通道图像的清晰度，确定车窗上存在第一类杂物。
若暗通道图像中灰度值未超过预设灰度阈值的像素的数量小于等于暗通道图像中灰度值超过预设灰度阈值的像素的数量，且暗通道图像的清晰度小于预设清晰度，则确定车窗上存在第一类杂物。
示例性的，以暗通道图像的清晰度为暗通道图像中的像素的方差为例，单帧车窗图像中包含有2*2个像素，则其对应的暗通道图像为包含有2*2个像素(这2*2个像素分别为A、B、C、D)的灰度图像，像素的灰度值的取值范围为0～255。在暗通道图像中，像素A的灰度值为7，像素B的灰度值为1，像素C的灰度值为3，像素D 的灰度值为5，则该暗通道图像的清晰度(即该暗通道图像中的像素的灰度值的方差)为[(7-4) 2+(1-4) 2+(3-4) 2+(5-4) 2]/4＝5。若预设灰度阈值为2，预设清晰度为6，像素B的灰度值未超过预设灰度阈值，像素A、C、D的灰度值超过预设灰度阈值，1<3，暗通道图像的清晰度5<6，则车窗上存在第一类杂物；若预设灰度阈值为3，预设清晰度为6，像素B、C的灰度值未超过预设灰度阈值，像素A、D的灰度值超过预设灰度阈值，2＝2，暗通道图像的清晰度5<6，则车窗上存在第一类杂物；若预设灰度阈值为2，预设清晰度为4，像素B的灰度值未超过预设灰度阈值，像素A、C、D的灰度值超过预设灰度阈值，1<3，暗通道图像的清晰度5>4，则车窗上不存在第一类杂物；若预设灰度阈值为5，预设清晰度为6，像素B、C、D的灰度值未超过预设灰度阈值，像素A的灰度值超过预设灰度阈值，3>1，暗通道图像的清晰度5<6，则车窗上不存在第一类杂物；若预设灰度阈值为5，预设清晰度为4，像素B、C、D的灰度值未超过预设灰度阈值，像素A的灰度值超过预设灰度阈值，3>1，暗通道图像的清晰度5>4，则车窗上不存在第一类杂物。
S404、根据连续i帧车窗图像中的像素的RGB值，确定车窗上存在第二类杂物。
其中，连续i帧车窗图像为上述步骤S401中摄像头多次拍摄得到的多帧车窗图像中的部分车窗图像，其中i为大于1的整数。
可选的，第二类杂物可以为灰尘、落叶、鸟的粪便等杂质。
先根据连续i帧车窗图像中的像素的RGB值，即连续i帧车窗图像中的像素在r、g、b三个颜色通道上的取值，确定这连续i帧车窗图像中同一位置的像素的RGB值的变化，建立图像背景模型。该图像背景模型中的像素表示连续i帧车窗图像中的像素的RGB值的变化。具体的，如果连续i帧车窗图像中同一位置的像素的RGB值未发生变化，则在图像背景模型中，与连续i帧车窗图像中未发生变化的像素相对应的像素的RGB值为(0，0，0)；若连续i帧车窗图像中同一位置的像素的RGB值发生变化，则在图像背景模型中与连续i帧车窗图像中发生变化的像素相对应的像素的RGB值为(0，0，0)之外的其他RGB值。通常，在车辆行驶过程中，若车窗上存在落叶等杂物时，车窗上除存在落叶外的位置的画面(像素取值)会随着车辆的行驶发生变化，而车窗上存在落叶的位置的画面(像素取值)不会随着车辆的行驶发生变化或者发生的变化程度较小，因此，可以根据图像背景模型中的像素的RGB值，确定连续i帧车窗图像中RGB值未发生变化的像素，或者连续i帧车窗图像中RGB值发生变化的程度未超过预设变化阈值的像素所对应的车窗区域为第一区域，第一区域内存在第二类杂物。
示例性的，每帧车窗图像中包括2*2个像素，以i＝3为例，在连续3帧车窗图像中，第1帧车窗图像中包括像素A1、B1、C1和D1，这4个像素的RGB值分别为(1，2，3)、(3，5，6)、(1，9，5)和(2，3，0)，第2帧车窗图像中包括像素A2、B2、C2和D2，这4个像素的RGB值分别为(3，2，5)、(7，5，2)、(7，1，9)和(8，5，2)，第3帧车窗图像中包括像素A3、B3、C3和D3，这4个像素的RGB值分别为(3，2，1)、(5，5，4)、(5，1，7)和(6，3，2)。两两计算这3帧车窗图像中同一位置的像素的RGB值的差值，并确定所得差值的平均值为图像背景模型中相应位置的像素的RGB值，则该图像背景模型中的像素A4、B4、C4和D4的 RGB值分别为(4/3，0，8/3)、(8/3，0，8/3)、(4，16/3，8/3)和(4，4/3，4/3)。若预设变化阈值为9，则对于上述连续2帧车窗图像对应的图像背景模型中的像素A4、B4、C4和D4来说，4/3+4/3<9，8/3+8/3<9，4+16/3+8/3>9，4+4/3+4/3<9，即图像背景模型中的像素A4、B4和D4所对应的连续3帧车窗图像中像素的RGB值发生变化，但发生变化的程度未超过预设变化阈值9，则图像背景模型中的像素A4、B4和D4对应的车窗区域，也即连续3帧车窗图像中的像素A1、B1和D1(或者说是像素A2、B2和D2，或者说是像素A3、B3和D3)所对应的车窗区域为第一区域，第一区域内存在第二类杂物。
可选的，在利用连续i帧车窗图像中的像素的RGB值建立图像背景模型时，先确定这连续i帧车窗图像中的同一位置的像素的RGB值的均值，再根据求得的这连续i帧车窗图像中每一位置上的像素的RGB值的均值，确定这连续i帧车窗图像所对应的均值图像。根据均值图像中的像素的RGB值与第i帧车窗图像中同一位置的像素的RGB值的差值，建立图像背景模型。
示例性的，每帧车窗图像中包括2*2个像素，以i＝2为例，在连续2帧车窗图像中，第1帧车窗图像中包括像素A1、B1、C1和D1，这4个像素的RGB值分别为(1，2，3)、(3，5，6)、(1，9，5)和(2，3，0)，第2帧车窗图像中包括像素A2、B2、C2和D2，这4个像素的RGB值分别为(3，2，5)、(7，5，2)、(7，1，9)和(8，5，2)。根据连续2帧车窗图像中同一位置的像素的RGB值，确定该均值图像中的像素A3、B3、C3和D3的RGB值分别为(2，2，4)、(5，5，4)、(4，5，7)和(5，4，1)，该均值图像中像素的RGB值与第2帧车窗图像中的像素的RGB值的差值为(1，0，1)、(2，0，2)、(3，4，2)和(3，1，1)，则根据这连续2帧车窗图像得到图像背景模型，该图像背景模型中的像素A4、B4、C4和D4的RGB值分别为(1，0，1)、(2，0，2)、(3，4，2)和(3，1，1)。
可选的，在根据图像背景模型中的像素的RGB值确定第一区域时，根据图像背景模型中的像素的RGB值，以及第一预设算法，确定连续i帧车窗图像中同一位置的像素的RGB值的变化，并进一步确定连续i帧车窗图像中RGB值未发生变化的像素所对应的车窗区域为第一区域，第一区域内存在第二类杂物。其中，第一预设算法为显著性检测算法或者目标检测算法。
示例性的，以i＝2为例，每帧车窗图像中包括2*2个像素，在连续2帧车窗图像中，第1帧车窗图像中包括像素A1、B1、C1和D1，第2帧车窗图像中包括像素A2、B2、C2和D2，连续2帧车窗图像对应的图像背景模型中包含像素A3、B3、C3和D3。根据显著性检测算法或者目标检测算法，确定图像背景模型中的像素C3和D3对应的连续2帧车窗图像中的像素C1和D1与C2和D2的RGB值未发生变化，因此图像背景模型中的像素C3和D3对应的车窗区域，也即连续2帧车窗图像中的像素C1和D1与C2和D2对应的车窗区域为第一区域，第一区域内存在第二类杂物。
需要说明的是，上述步骤S402-步骤S403用于检测车窗上的第一类杂物，步骤S404用于检测车窗上是否存在第二类杂物。其中，检测车窗上是否存在第一类杂物与检测车窗上是否存在第二类杂物的顺序并不限定，即可以先执行步骤S402和步骤S403，再执行步骤S404，也可以先执行步骤S404，再执行步骤S402和步骤S403，也可以同时 执行步骤S402和步骤S403与步骤S404。
S405、控制清洁工具去除杂物。
其中，所述清洁工具包括第一清洁工具以及第二清洁工具。第一清洁工具主要用于通过除湿、调节温度等操作去除第一类杂物，例如空调等。第二清洁工具主要用于去除附着于车辆挡风玻璃上的雨点及灰尘等第二类杂物，以改善驾驶人的能见度，例如雨刷等。下文分别介绍去除第一类杂物和第二类杂物的具体实现方式。
(1)控制清洁工具去除第一类杂物。
可选的，根据车内外温度数据控制第一清洁工具开启预设时长，去除第一类杂物，其中，预设时长可以是根据车内外温度数据确定的，也可以是驾驶员自行确定的。
示例性的，以第一清洁工具为空调，第一类杂物为雾为例，若车内温度T1大于车外温度T2，则根据车内外温度数据设置空调吹风温度为T2以及空调吹风时长为S1，使得空调以吹风温度T2开启，并在开启时长S1后关闭，以降低车窗温度，清洁车窗上的雾；若车内温度T1小于车外温度T2，则空调吹风温度为T2，则根据车内外温度数据设置空调吹风温度为T2以及空调吹风时长为S2，使得空调以吹风温度T2开启，并在开启时长S2后关闭，以提高车窗温度，清洁车窗上的雾。
可选的，在控制清洁工具去除第一类杂物后，重新进行步骤S401-步骤S403，检测车窗上是否还存在第一类杂物，若是，则重新控制清洁工具去除第一类杂物。
(2)控制清洁工具去除第二类杂物。
可选的，在一种可能的实现方式中，控制第二清洁工具以预设工作频率工作预设时长，以清洁第一区域内的第二类杂物。其中，第二清洁工具的预设工作频率以及工作的预设时长可以根据第二清洁工具的清洁效果预先设定，也可以由驾驶员自行设定。
在不同的行驶状态或者不同的驾驶员状态下，对第二清洁工具的工作时长以及工作频率进行适应性的调整，可以最小程度的减少车窗清洁对驾驶员造成的影响，保证车辆驾驶过程中的安全性。
因此，可选的，在另一种可能的实现方式中，在控制第二清洁工具清洁第以区域内的第二类杂物时，可以先根据行驶状态和/或驾驶员状态、行驶状态和/或驾驶员状态对应的第一预设权重参数，以及第二清洁工具连续工作的最大时长，确定第二清洁工具工作时长。再根据行驶状态和/或驾驶员状态、行驶状态和/或驾驶员状态对应的第二预设权重参数，以及第二清洁工具工作的最大频率，确定第二清洁工具工作频率，最后根据第二清洁工具工作时长以及第二清洁工具工作频率控制第二清洁工具清洁第一区域，去除第二类杂物。其中，上述行驶状态可以为车辆的驾驶速度，包括高速、中速和低速，上述驾驶员状态可以为驾驶员的疲劳程度，包括重度疲劳、轻度疲劳以及不疲劳，以下不再赘述。
需要说明的是，第一预设权重参数和第二预设权重参数可以预先设定，也可以是驾驶员自行设定。
第二清洁工具工作时长可以根据行驶状态确定，也可以根据驾驶员状态确定，也可以结合这两个因素一起确定。
示例性的，根据行驶状态、驾驶员状态、行驶状态对应的第一预设权重参数、驾 驶员状态对应的第一预设权重参数、以及第二预设算法
Figure PCTCN2020102184-appb-000011
确定第二清洁工具工作时长。其中，t 1为第二清洁工具工作时长，t s-max为第二清洁工具连续工作的最大时长，e为自然对数的底数，α 1和β 1分别为行驶状态和驾驶员状态对应的第一预设权重参数，α 1+β 1＝1，s 1表示行驶状态，s 2表示驾驶员状态。示例性的，行驶状态s 1的取值为-1、-3、-10，分别代表驾驶速度为高速、中速、低速，驾驶员状态s 2的取值为-1、-3、-10，分别代表驾驶员状态为重度疲劳、轻度疲劳、不疲劳。s 1的取值越大，驾驶速度越大，或者s 2的取值越大，驾驶员越疲劳，则第二清洁工具工作时长t 1越短。
或者，根据行驶状态、行驶状态对应的第一预设权重参数、以及第二预设算法
Figure PCTCN2020102184-appb-000012
确定第二清洁工具工作时长，其中，t 1为第二清洁工具工作时长，t s-max为第二清洁工具连续工作的最大时长，e为自然对数的底数，α 1为行驶状态对应的第一预设权重参数，α 1＝1，s 1表示行驶状态，s 1的取值参考前文所述。
或者，根据驾驶员状态、驾驶员状态对应的第一预设权重参数、以及第二预设算法
Figure PCTCN2020102184-appb-000013
确定第二清洁工具工作时长，其中，t 1为所述第二清洁工具工作时长，t s-max为所述第二清洁工具连续工作的最大时长，e为自然对数的底数，β 1为驾驶员状态对应的第一预设权重参数，β 1＝1，s 2表示驾驶员状态，s 2的取值参考前文所述。
同理，第二清洁工具工作频率可以根据行驶状态确定，也可以根据驾驶员状态确定，也可以结合这两个因素一起确定。
示例性的，根据行驶状态、驾驶员状态、行驶状态对应的第二预设权重参数、驾驶员状态对应的第二预设权重参数、以及第三预设算法
Figure PCTCN2020102184-appb-000014
确定第二清洁工具工作频率。其中，f为第二清洁工具工作频率，f max为第二清洁工具工作的最大频率，e为自然对数的底数，α 2和β 2分别为行驶状态和驾驶员状态对应的第二预设权重参数，α 2+β 2＝1，s 1表示行驶状态，s 2表示驾驶员状态。示例性的，行驶状态s 1的取值为-1、-3、-10，分别代表驾驶速度为高速、中速、低速，驾驶员状态s 2的取值为-1、-3、-10，分别代表驾驶员状态为重度疲劳、轻度疲劳、不疲劳。s 1的取值越大，驾驶速度越大，或者s 2的取值越大，驾驶员越疲劳，则第二清洁工具工作频率f越大。
或者，根据行驶状态、行驶状态对应的第二预设权重参数、以及第三预设算法
Figure PCTCN2020102184-appb-000015
确定第二清洁工具工作频率。其中，f为第二清洁工具工作频率，f max为第二清洁工具工作的最大频率，e为自然对数的底数，α 2为行驶状态对应的第二预设权重参数，α 2＝1，s 1表示行驶状态，s 1的取值参考前文所述。
或者，根据驾驶员状态、驾驶员状态对应的第二预设权重参数、以及第三预设算法
Figure PCTCN2020102184-appb-000016
确定第二清洁工具工作频率。其中，f为第二清洁工具工作频率，f max为第二清洁工具工作的最大频率，e为自然对数的底数，β 2分别为驾驶员状态对应的第二预设权重参数，β 2＝1，s 2表示驾驶员状态，s 2的取值参考前文所述。
需要说明的是，示例性的，在第二预设算法以及第三预设算法中，行驶状态s 1的取值和驾驶员状态s 2的取值可以参考前文所述。当s 1＝-1(或s 2＝-1)时，t 1较短，f较大，可以实现快速除杂，减少视线遮挡，从而提高驾驶员在高速驾驶过程中的安全 性；当s 1＝-3(和/或s 2＝-3)时，t 1和f适中；当s 1＝-10(或s 2＝-10)时，t 1较长，f较小，可以减少第二清洁工具频率过大带来的不适感，同时第二清洁工具的工作时间变长，因此可以保证车窗的清洁效率。
(3)控制清洁工具去除第一类杂物和第二类杂物。
需要说明的是，若检测到车窗上既存在第一类杂物也存在第二类杂物，则需要控制清洁工具去除第一类杂物和第二类杂物。其中，控制清洁工具去除第一类杂物和控制清洁工具去除第二类杂物的顺序并不限定，可先控制清洁工具去除第一类杂物，再控制清洁工具去除第二类杂物，也可以先控制清洁工具去除第二类杂物，再控制清洁工具去除第一类杂物，或者也可以控制清洁工具同时去除第一类杂物和第二类杂物。
具体的，控制清洁工具去除第一类杂物和控制清洁工具去除第二类杂物的具体实施方式可以参见本步骤上述(1)和(2)所述的实施方式。
在本申请所描述的实施例所描述的车窗自动清洁方法中，利用单帧车窗图像对应的暗通道图像中的像素的灰度值，对车窗上是否存在第一类杂物进行检测，和/或利用连续i帧车窗图像中的像素的RGB值对车窗上是否存在第二类杂物进行检测。若车窗上存在第一类杂物和/或第二类杂物，控制清洁工具进行除杂。首先，在进行杂物检测后进行清洁，可以减少盲目清洁，提高清洁效率，节省清洁资源。另外，通过上述过程，本申请实施例可以实现车窗的自动清洁，减少人工操作，提高车辆驾驶过程中的安全性。
为了进一步提高检测车窗上是否存在第二类杂物的准确性，本申请还提供了一种车窗自动清洁方法，在通过上述步骤S404确定第一区域之后，所述方法还包括如下步骤S601-S602，下面结合图6对本申请的实施例进行描述：
S601、在连续j帧车窗图像中的每一帧车窗图像上确定与第一区域相对应的第二区域，得到j个第二区域。
其中，连续j帧车窗图像为上述步骤S401中摄像头多次拍摄得到的多帧车窗图像中的部分车窗图像，或者连续j帧车窗图像为摄像头再次进行多次拍摄后得到的多帧车窗图像中的部分或全部车窗图像，j为大于1的整数，且连续i帧车窗图像与连续j帧车窗图像之间可能包括其他至少一帧车窗图像，也可能不包括其他帧的车窗图像。另外，所述第二区域中的所有像素对应的车窗区域即为第一区域。
可选的，在一种可能的实现方式中，确定第一区域之后，先获取连续i帧车窗图像之后的连续j帧车窗图像，再根据第一区域所对应的车窗图像中的像素位置，确定连续j帧车窗图像中每一帧车窗图像中的第二区域，得到j个第二区域。
S602、若j个第二区域内同一位置的像素的RGB值未发生变化，则确定第一区域内存在杂物。
在车辆行驶过程中，若车窗上的第一区域内存在落叶等第二类杂物，则车窗上存在落叶等第二类杂物的第一区域的画面(像素取值)不会随着车辆的行驶发生变化或者发生的变化程度较小，因此，若j个第二区域内同一位置的像素的RGB值未发生变化，或者j个第二区域内同一位置的RGB值发生变化的程度未超过预设变化阈值，则确定第一区域内确实存在第二类杂物。
示例性的，每帧车窗图像中包括2*2个像素，以j＝2为例，在连续j＝2帧车窗图 像中，第1帧车窗图像中包括像素A1、B1、C1和D1，第2帧车窗图像中包括像素A2、B2、C2和D2。所述第1帧车窗图像中与第一区域相对应的第二区域为E1，E1中包括像素A1和B1，这两个像素的RGB值分别为(1，2，3)和(3，5，6)。所述第2帧车窗图像中与第一区域相对应的第二区域为E2，E2中包括像素A2和B2，这两个像素的RGB值分别为(3，2，5)和(7，5，2)。计算这2个第二区域中同一位置的像素的RGB值的差值，即像素A1和像素A2的RGB值的差值(2，0，2)，以及像素B1和像素B2的差值(4，0，4)。若预设变化阈值为2，则在j＝2个第二区域中，对于像素A1和像素A2的RGB值的差值来说，2+2>2，对于像素B1和B2的RGB值的差值来说，4+4>2，即j＝2个第二区域内同一位置的像素的RGB值发生变化，但发生变化的程度未超过预设变化阈值2，因此可以确定在本示例中第一区域内存在第二类杂物。
可选地，在确定j个第二区域内同一位置的像素的RGB值发生变化时，可以先确定与连续i帧车窗图像相对应的图像背景模型中与第一区域相对应的第三区域，再确定每个第二区域中像素的RGB值与第三区域中同一位置的像素的RGB值的差值，将得到的差值并进行累加，得到累加值。若累加值超过预设阈值，则确定j个第二区域内同一位置的像素的RGB值发生变化；若累加值不超过预设阈值，则确定j个第二区域内同一位置的像素的RGB值未发生变化。
示例性的，以j＝2为例，第1个第二区域中包括像素A1和像素B1，这两个像素的RGB值分别为(1，2，3)和(3，5，6)，第2个第二区域内包括像素A2和像素B2，这两个像素的RGB值分别为(3，2，5)和(7，2，5)。在确定j＝2个第二区域后，根据第一区域对应的车窗区域，确定第一区域在连续i帧车窗图像对应的图像背景模型中对应的第三区域，第三区域中包括像素A3和B3，这两个像素的RGB值分别为(2，0，2)和(4，3，1)。根据h＝∑ k||F 0-f k|| χ，确定第二区域中的每个像素的RGB值与第三区域中的同一位置的像素的RGB值的差值的累加值，其中，F 0表示第三区域，f k表示连续j帧车窗图像中第k帧车窗图像对应的第二区域，k为大于0的整数，||.|| χ为距离度量函数，在本示例中，其度量算法为绝对值算法。因此可以确定第二区域中像素A1、A2的RGB值与第三区域中的像素A3的RGB值的差值的绝对值并分别为(1，2，1)和(1，2，5)，第二区域中的像素B1和B2的RGB值与第三区域中的像素B3的差值分别为(1，2，5)和(3，1，4)，对这几个差值进行累加，得到累加值h为(6，7，15)。若预设阈值为0，6+7+15>0，则确定本示例中j＝2个第二区域内同一位置的像素的RGB值发生变化；若预设阈值为30，6+7+15<30，则确定本示例中j＝2个第二区域内同一位置的像素的RGB值未发生变化。
可选的，在另一种可能的实现方式中，在确定j个第二区域内同一位置的像素的RGB值发生变化时，可以先确定与连续i帧车窗图像相对应的图像背景模型中与第一区域相对应的第三区域，再确定与连续j帧车窗图像相对应的图像背景模型中与第一区域相对应的第四区域，计算第三区域中的像素的RGB值与第四区域中的像素的RGB值的差值，将得到的差值进行累加得到累加值。若累加值超过预设阈值，则确定j个第二区域内同一位置的像素的RGB值发生变化；若累加值不超过预设阈值，则确定j个第二区域内同一位置的像素的RGB值未发生变化。
示例性的，以j＝2为例，第1个第二区域中包括像素A1和像素B1，这两个像素 的RGB值分别为(1，2，3)和(3，5，6)，第2个第二区域内包括像素A2和像素B2，这两个像素的RGB值分别为(3，2，5)和(7，3，6)。在确定j＝2个第二区域后，根据第一区域对应的车窗区域，确定第一区域在连续i帧车窗图像对应的图像背景模型中对应的第三区域，第三区域中包括像素A3和B3，这两个像素的RGB值分别为(1，0，1)和(4，3，1)。再确定第一区域在连续j＝2帧车窗图像对应的图像背景模型中对应的第四区域，第四区域中包括像素A4和B4，这两个像素的RGB值分别为(1，0，1)和(2，1，0)。根据h＝∑ k||F 0-f k|| χ，确定第三区域中的像素的RGB值与第四区域中的同一位置的像素的RGB值的汉明距离的累加值，其中，F 0表示第三区域，f k表示第一区域在连续j帧车窗图像的图像背景模型中对应的第四区域，k＝j，||.|| χ为距离度量函数，在本示例中，其度量算法为汉明距离算法。第三区域中像素A3的RGB值与第四区域中的像素A4的RGB值相同，则第三区域中像素A3与第四区域中的像素A4的汉明距离为0，第三区域中像素A4的RGB值与第四区域中的像素B4的RGB值不同，则第三区域中像素B3与第四区域中的像素B4的汉明距离为1。因此，第三区域与第四区域的汉明距离为1，h＝1。若预设阈值为0，则确定本示例中j＝2个第二区域内同一位置的像素的RGB值发生变化；若预设阈值为2，则确定本示例中j＝2个第二区域内同一位置的像素的RGB值未发生变化。
需要说明的是，相比于绝对值算法，利用汉明距离度量算法进行数据处理，可以提升数据处理的速度，若汉明距离用二进制来表示，则可以进一步加快数据处理的速度，从而提高确定第二类杂物所在区域的效率。当然，汉明距离的具体表示形式还可以根据实际需求确定为八进制或者十六进制等，并不局限于上述二进制或者十进制。可选的，除了绝对值算法与汉明距离之外，||.|| χ所表示的距离度量函数的度量算法还可以为欧式距离算法等。
可选的，在另一种可能的实现方式中，在确定j个第二区域内同一位置的像素的RGB值发生变化时，可以先确定与连续i帧车窗图像相对应的均值图像中与第一区域相对应的第五区域。再确定每个第二区域中像素的RGB值与第五区域中同一位置的像素的RGB值的差值，将差值进行累加得到累加值。若累加值超过预设阈值，则确定j个第二区域内同一位置的像素的RGB值发生变化；若累加值不超过预设阈值，则确定j个第二区域内同一位置的像素的RGB值未发生变化。
在本申请实施例中，在利用连续i帧车窗图像确定第一区域之后，又根据连续i帧车窗图像之后的连续j帧车窗图像确定j个第二区域，并根据这j个第二区域，验证所述第一区域内是否确实存在第二类杂物，以提高确定车窗上存在第二类杂物的准确性，并进一步提高车窗自动清洁的效率。
为了更好的清洁第一区域内存在的第二类杂物，保证自动清洁的良好效果，本申请还提供了一种车窗自动清洁方法，在控制清洁工具去除第二类杂物时，所述清洁工具还可以包括第三清洁工具，其中，第三清洁工具主要用于清洗车窗，例如玻璃水等。因此，在根据步骤405的具体实施方式(2)和具体实施方式(3)，利用第二清洁工具对第一区域内的第二类杂物进行清洁后，本申请还可以结合第三清洁工具来去除第一区域内的第二类杂物，具体的，所述方法还包括步骤S701-S703，下面结合图7对本申请的实施例进行描述：
S701、判断车窗上的第一区域内的第二类杂物是否被清洁。
其中，所述第一区域为步骤S404中确定的第一区域，也即图6所述的方法中的 第一区域。
具体的，本步骤S701的具体实施方式可以参见上述实施例中步骤S404所述的实时方式。
可选的，在控制第二清洁工具去除第一区域内的第二类杂物后，执行步骤S701，判断车窗上的第一区域内的第二类杂物是否被清洁。若车窗上的第一区域内的第二类杂物已经被清洁，则结束车窗自动清洁过程，并保持车内摄像头开启，对车窗进行实时监控；若车窗上的第一区域内的第二类杂物未被清洁(即车窗上的第一区域内仍存在第二类杂物)，则执行下述步骤S702。
S702、控制第三清洁工具去除第二类杂物。
可选的，在确定车窗上的第一区域内的第二类杂物未被清洁，则可以控制第三清洁工具以预设工作间隔工作预设时长，以清洁第一区域内的第二类杂物。其中，第三清洁工具的预设工作间隔以及工作的预设时长可以根据第三清洁工具的清洁效果预先设定，也可以由驾驶员自行设定。可选的，第三清洁工具可以为清洁液，第三清洁工具工作时长即清洁液喷洒时长。
在不同的行驶状态或者不同的驾驶员状态下，对第三清洁工具的工作时长以及工作间隔进行适应性的调整，可以最小程度的减少车窗清洁对驾驶员造成的影响，保证车辆驾驶过程中的安全性。因此，在一种可能的实现方式中，在控制第三清洁工具清洁第一区域内的第二类杂物时，可以先根据行驶状态和/或驾驶员状态、行驶状态和/或驾驶员状态对应的第三预设权重参数、以及第三清洁工具连续工作的最大时长，确定第三清洁工具工作时长。再根据行驶状态和/或驾驶员状态、行驶状态和/或驾驶员状态对应的第四预设权重参数，以及第三清洁工具工作的最大间隔，确定第三清洁工具工作间隔。最后根据第三清洁工具工作时长以及第三清洁工具工作间隔控制第三清洁工具清洁第一区域，再根据第二清洁工具工作时长以及第二清洁工具工作频率控制第二清洁工具清洁第一区域，以去除第二类杂物。可选的，第三清洁工具可以为清洁液，第三清洁工具工作时长即清洁液喷洒时长。另外，上述行驶状态可以为车辆的驾驶速度，包括高速、中速和低速，上述驾驶员状态可以为驾驶员的疲劳程度，包括重度疲劳、轻度疲劳以及不疲劳，以下不再赘述。
需要说明的是，第三预设权重参数和第四预设权重参数可以是预先设定的，也可以是驾驶员自行设定的。
第三清洁工具工作时长可以根据行驶状态确定，也可以根据驾驶员状态确定，也可以结合这两个因素一起确定。
示例性的，根据行驶状态、驾驶员状态、行驶状态对应的第三预设权重参数、驾驶员状态对应的第三预设权重参数、以及第四预设算法
Figure PCTCN2020102184-appb-000017
确定第三清洁工具工作时长。其中，t 2为第三清洁工具工作时长，t l-max为第三清洁工具连续工作的最大时长，e为自然对数的底数，α 3和β 3分别为行驶状态和驾驶员状态对应的第三预设权重参数，α 3+β 3＝1，s 1表示行驶状态，s 2表示驾驶员状态。示例性的，行驶状态s 1的取值为-1、-3、-10，分别代表驾驶速度为高速、中速、低速，驾驶员状态s 2的取值为-1、-3、-10，分别代表驾驶员状态为重度疲劳、轻度疲劳、不疲劳。s 1的取值越大，驾驶速度越大，或者s 2的取值越大，驾驶员越疲劳，则第三清洁工具工作 时长t 2越短。
或者，根据行驶状态、行驶状态对应的第三预设权重参数、以及第四预设算法
Figure PCTCN2020102184-appb-000018
确定第三清洁工具工作时长。其中，t 2为第三清洁工具工作时长，t l-max为第三清洁工具连续工作的最大时长，e为自然对数的底数，α 3为行驶状态对应的第三预设权重参数，α 3＝1，s 1表示行驶状态，s 1的取值参考前文所述。
或者，根据驾驶员状态、驾驶员状态对应的第三预设权重参数、以及第四预设算法
Figure PCTCN2020102184-appb-000019
确定第三清洁工具工作时长。其中，t 2为第三清洁工具工作时长，t l-max为第三清洁工具连续工作的最大时长，e为自然对数的底数，β 3为驾驶员状态对应的第三预设权重参数，β 3＝1，s 2表示驾驶员状态，s 2的取值参考前文所述。
同理，第三清洁工具工作间隔可以根据行驶状态确定，也可以根据驾驶员状态确定，也可以结合这两个因素一起确定。
示例性的，根据行驶状态、驾驶员状态、行驶状态对应的第四预设权重参数、驾驶员状态对应的第四预设权重参数、以及第五预设算法
Figure PCTCN2020102184-appb-000020
确定第三清洁工具工作间隔，其中，△t为第三清洁工具工作间隔，△t max为第三清洁工具工作的最大间隔，e为自然对数的底数，α 4和β 4分别为行驶状态和驾驶员状态对应的第四预设权重参数，α 4+β 4＝1，s 1表示行驶状态，s 2表示驾驶员状态。示例性的，行驶状态s 1的取值为-1、-3、-10，分别代表驾驶速度为高速、中速、低速，驾驶员状态s 2的取值为-1、-3、-10，分别代表驾驶员状态为重度疲劳、轻度疲劳、不疲劳。s 1的取值越大，驾驶速度越大，或者s 2的取值越大，驾驶员越疲劳，则第三清洁工具工作间隔△t越短。
或者，根据行驶状态、驾驶员状态、行驶状态对应的第四预设权重参数、驾驶员状态对应的第四预设权重参数、以及第五预设算法
Figure PCTCN2020102184-appb-000021
确定第三清洁工具工作间隔，其中，△t为第三清洁工具工作间隔，△t max为第三清洁工具工作的最大间隔，e为自然对数底数，α 4为驾驶员状态对应的第四预设权重参数，α 4＝1，s 1表示行驶状态，s 1的取值参考前文所述。
或者，根据驾驶员状态、驾驶员状态对应的第四预设权重参数、以及第五预设算法
Figure PCTCN2020102184-appb-000022
确定第三清洁工具工作间隔，其中，△t为第三清洁工具工作间隔，△t max为第三清洁工具工作的最大间隔，e为自然对数的底数，β 4为驾驶员状态对应的第四预设权重参数，β 4＝1，s 2表示驾驶员状态，s 2的取值参考前文所述。
需要说明的是，示例性的，在第四预设算法以及第五预设算法中，行驶状态s 1的取值和驾驶员状态s 2的取值可以参考前文所述。当s 1＝-1(或s 2＝-1)时，t 2较短，△t较大，减少清洁过程中由第三清洁工具工作间隔较小造成的不适感以及视线遮挡，从而提高驾驶员在高速驾驶过程中的安全性；当s 1＝-3(和/或s 2＝-3)时，t 2和△t适中；当s 1＝-10(或s 2＝-10)时，t 2较长，△t较小，第三清洁工具的工作时间变长，可以保证车窗清洁。
S703、控制第二清洁工具去除第二类杂物。
在根据第三清洁工具工作时长以及第三清洁工具工作间隔控制第三清洁工具清洁第一区域后，再次根据第二清洁工具工作时长以及第二清洁工具工作频率控制第二清洁工具清洁第一区域，以去除第一区域内的第二类杂物。具体的，控制第二清洁工 具去除第一区域内的第二类杂物的具体实施方式可以参见上述实施例的步骤S405的(2)所述的实施方式，在此不再赘述。
需要说明的是，在上述步骤S703结束后，还需要重新执行步骤S701，以判断车窗上的第二类杂物是否被清洁。若车窗上的第一区域内的第二类杂物已经被清洁，则结束车窗自动清洁过程，并保持车内摄像头开启，对车窗进行实时监控；若车窗上的第一区域内的第二类杂物未被清洁(即车窗上的第一区域内仍存在第二类杂物)，则根据第二清洁工具工作时长、第二清洁工具工作频率、第三清洁工具工作时长以及第三清洁工具工作间隔，完成q次控制第二清洁工具和第三清洁工具去除第二类杂物的操作之后，结束车窗自动清洁过程，并保持车内摄像头开启，实时监控车窗。其中，q可以由驾驶员自行确定，也可以根据第二清洁工具和第三清洁工具的清洁效果预先设定。通过上述过程，本申请实施例对车窗上存在的未被清洁的第二类杂物进行多次自动清洁，可以尽可能的去除车窗上存在的第二类杂物，以实现较好的自动清洁效果。
在本申请实施例中，在控制第二清洁工具清洁第一区域内的第二类杂物后，先判断第一区域内的第二类杂物是否被清洁，若未被清洁，则控制第三清洁工具以及第二清洁工具清洁第二类杂物。通过多种清洁工具的相配合，本申请实施例可以尽可能的去除车窗上存在的第二类杂物，从而实现较好的自动清洁效果。
本申请实施例可以根据上述方法示例对车窗自动清洁装置进行功能模块的划分，在采用对应各个功能划分各个功能模块的情况下，图8示出上述实施例中所涉及的车窗自动清洁装置的一种可能的结构示意图。如图8所示，车窗自动清洁装置包括获取单元801、确定单元802、控制单元803。当然，车窗自动清洁装置还可以包括其他模块，或者车窗自动清洁装置可以包括更少的模块。
获取单元801，用于获取车窗图像。
确定单元802，用于确定单帧车窗图像对应的暗通道图像。
确定单元802，还用于根据暗通道图像中灰度值超过预设灰度阈值的像素的数量和/或暗通道图像的清晰度，确定车窗上存在第一类杂物。
具体的，确定单元802，用于在暗通道图像中灰度值未超过预设灰度阈值的像素的数量小于等于暗通道图像中灰度值超过预设灰度阈值的像素的数量，和/或暗通道图像的清晰度小于预设清晰度时，确定车窗上存在第一类杂物。其中，暗通道图像的清晰度为暗通道图像中像素的灰度值的方差。
和/或确定单元802，还用于根据连续i帧车窗图像中的像素的RGB值，确定车窗上存在第二类杂物。
具体的，确定单元802，还用于先根据连续i帧车窗图像中像素的RGB值，建立图像背景模型。根据图像背景模型中的像素的RGB值，确定第一区域，进而确定第一区域内存在第二类杂物。其中，图像背景模型中的像素的RGB值表示连续i帧车窗图像中同一位置的像素的RGB值的变化，i为大于1的整数，第一区域为连续i帧车窗图像中RGB值未发生变化的像素对应的车窗区域。
示例性的，确定单元802，还用于根据连续i帧车窗图像中同一位置的像素的RGB值的均值，确定连续i帧车窗图像的均值图像，根据连续i帧车窗图像的均值图像中的像素的RGB值与第i帧车窗图像中的像素的RGB值的差值，建立图像背景模型。
示例性的，确定单元802，还用于图像背景模型中的像素的RGB值以及第一预设算法，确定第一区域。其中，第一预设算法为显著性检测算法或者目标检测算法。
示例性的，确定单元802，还用于确定图像背景模型中与第一区域相对应的第三区域，确定每个第二区域中像素的RGB值与第三区域中同一位置的像素的RGB值的差值并进行累加，得到累加值。若累加值超过预设阈值，则确定j个第二区域内同一位置的像素的RGB值发生变化。
可选的，在另一种可能的实现方式中，确定单元802在确定第一区域后，先在连续j帧车窗图像中的每一帧车窗图像上确定与第一区域相对应的一个第二区域，得到j个第二区域。若j个第二区域内同一位置的像素的RGB值未发生变化，则确定第一区域内存在第二类杂物。其中，连续j帧车窗图像位于连续i帧车窗图像之后，j为大于1的整数。
控制单元803，用于控制清洁工具去除杂物。其中，所述杂物包括第一类杂物和/或第二类杂物。
可选的，控制单元803，用于根据车内外温度数据控制第一清洁工具开启预设时长，去除第一类杂物。
和/或控制单元803，用于根据第二清洁工具工作时长以及第二清洁工具工作频率控制第二清洁工具清洁第一区域，去除第二类杂物。其中，第二清洁工具工作时长根据行驶状态和/或驾驶员状态、行驶状态和/或驾驶员状态对应的第一预设权重参数、以及第二清洁工具连续工作的最大时长确定，第二清洁工具工作频率根据行驶状态和/或驾驶员状态、行驶状态和/或驾驶员状态对应的第二预设权重参数，以及第二清洁工具工作的最大频率确定。
可选地，控制单元803，还用于根据行驶状态、驾驶员状态、行驶状态对应的第一预设权重参数、驾驶员状态对应的第一预设权重参数、第二清洁工具连续工作的最大时长以及第二预设算法，确定第二清洁工具工作时长，第二预设算法为
Figure PCTCN2020102184-appb-000023
控制单元803，还用于根据行驶状态、驾驶员状态、行驶状态对应的第二预设权重参数、驾驶员状态对应的第二预设权重参数、第二清洁工具工作的最大频率以及第三预设算法，确定第二清洁工具工作频率，第三预设算法为
Figure PCTCN2020102184-appb-000024
其中，t 1为第二清洁工具工作时长，t s-max为第二清洁工具连续工作的最大时长，f为第二清洁工具工作频率，f max为第二清洁工具工作的最大频率，e为自然对数的底数，α 1和β 1分别为行驶状态和驾驶员状态对应的第一预设权重参数，α 1+β 1＝1，α 2和β 2分别为行驶状态和驾驶员状态对应的第二预设权重参数，α 2+β 2＝1，s 1表示行驶状态、s 2表示驾驶员状态。
可选的，控制单元803，还用于先根据行驶状态和/或驾驶员状态、行驶状态和/或驾驶员状态对应的第三预设权重参数，以及第三清洁工具连续工作的最大时长，确定第三清洁工具工作时长。然后根据行驶状态和/或驾驶员状态、行驶状态和/或驾驶员状态对应的第四预设权重参数，以及第三清洁工具工作的最大间隔，确定第三清洁工具工作间隔。最后根据第三清洁工具工作时长以及第三清洁工具工作间隔控制第三清洁工具清洁第一区域，并根据第二清洁工具工作时长以及第二清洁工具工作频率控制第二清洁工具清洁第一区域，去除第二类杂物。
具体的，控制单元803，用于根据行驶状态、驾驶员状态、行驶状态对应的第三预设权重参数、驾驶员状态对应的第三预设权重参数、第三清洁工具连续工作的最大时长以及第四预设算法，确定第三清洁工具工作时长，第四预设算法为
Figure PCTCN2020102184-appb-000025
控制单元803，还用于根据行驶状态、驾驶员状态、行驶状态对应的第四预设权重参数、驾驶员状态对应的第四预设权重参数、第三清洁工具工作的最大间隔以及第五预设算法，确定第三清洁工具工作间隔，第五预设算法为
Figure PCTCN2020102184-appb-000026
其中，t 2为第三清洁工具工作时长，t l-max为第三清洁工具连续工作的最大时长，△t为第三清洁工具工作间隔，△t max为第三清洁工具工作的最大间隔，e为自然对数的底数，α 3和β 3分别为行驶状态和驾驶员状态对应的第三预设权重参数，α 3+β 3＝1，α 4和β 4分别为行驶状态和驾驶员状态对应的第四预设权重参数，α 4+β 4＝1，s 1表示行驶状态、s 2表示驾驶员状态。
参见图9，本申请还提供一种车窗自动清洁装置，包括处理器910以及存储器920。
处理器910与存储器920相连接(如通过总线940相互连接)。
可选的，车窗自动清洁装置还可包括收发器930，收发器930连接处理器910和存储器920，收发器用于接收/发送数据。
处理器910，可以执行图4、图6，以及图7所对应的任意一个实施方案及其各种可行的实施方式的操作。比如，用于执行获取单元801、确定单元802、控制单元803的操作，和/或本申请实施例中所描述的其他操作。
关于处理器、存储器、总线和收发器的具体介绍，可参见上文，这里不再赘述。
本申请还提供一种自动车窗清洁装置，包括非易失性存储介质，以及中央处理器，非易失性存储介质存储有可执行程序，中央处理器与非易失性存储介质连接，并执行可执行程序以实现本申请实施例如图4、图6或图7所示的自动车窗清洁方法。
本申请另一实施例还提供一种计算机可读存储介质，该计算机可读存储介质包括一个或多个程序代码，该一个或多个程序包括指令，当处理器在执行该程序代码时，该自动车窗清洁装置执行如图4、图6或图7所示的自动车窗清洁方法。
在本申请的另一实施例中，还提供一种计算机程序产品，该计算机程序产品包括计算机执行指令，该计算机执行指令存储在计算机可读存储介质中。自动车窗清洁装置的至少一个处理器可以从计算机可读存储介质读取该计算机执行指令，至少一个处理器执行该计算机执行指令使得自动车窗清洁装置实施执行图4、图6或图7所示的自动车窗清洁方法中相应步骤。
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的系统、装置和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
所述功能如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(read-only memory，ROM)、随机存取存储器(random access memory，RAM)、磁碟或者光盘 等各种可以存储程序代码的介质。
以上所述，仅为本申请的具体实施方式，但本申请的保护范围并不局限于此，任何在本申请揭露的技术范围内的变化或替换，都应涵盖在本申请的保护范围之内。

Claims (25)

  1. 一种车窗自动清洁方法，其特征在于，包括：
    获取车窗图像；
    确定单帧车窗图像对应的暗通道图像；
    根据所述暗通道图像中灰度值超过预设灰度阈值的像素的数量和/或所述暗通道图像的清晰度，确定车窗上存在第一类杂物；
    和/或根据连续i帧车窗图像中的像素的RGB值，确定车窗上存在第二类杂物；
    控制清洁工具去除杂物，所述杂物包括所述第一类杂物和/或第二类杂物。
  2. 根据权利要求1所述的车窗自动清洁方法，其特征在于，所述根据连续i帧车窗图像中的像素的RGB值，确定车窗上存在第二类杂物，具体包括：
    根据所述连续i帧车窗图像中像素的RGB值，建立图像背景模型；其中，所述图像背景模型中的像素的RGB值表示所述连续i帧车窗图像中同一位置的像素的RGB值的变化，i为大于1的整数；
    根据所述图像背景模型中的像素的RGB值，确定第一区域，所述第一区域为所述连续i帧车窗图像中RGB值未发生变化的像素对应的车窗区域；
    确定所述第一区域内存在第二类杂物。
  3. 根据权利要求2所述的车窗自动清洁方法，其特征在于，在所述根据所述图像背景模型中的像素的RGB值，确定第一区域之后，所述方法具体还包括：
    在连续j帧车窗图像中的每一帧车窗图像上确定与所述第一区域相对应的一个第二区域，得到j个第二区域；所述连续j帧车窗图像位于所述连续i帧车窗图像之后，j为大于1的整数；
    若所述j个第二区域内同一位置的像素的RGB值未发生变化，则确定所述第一区域内存在第二类杂物。
  4. 根据权利要求1-3任一项所述的车窗自动清洁方法，其特征在于，所述根据所述暗通道图像中灰度值超过预设灰度阈值的像素的数量和/或所述暗通道图像的清晰度，确定车窗上存在第一类杂物，具体包括：
    若所述暗通道图像中灰度值未超过预设灰度阈值的像素的数量小于等于所述暗通道图像中灰度值超过预设灰度阈值的像素的数量，和/或所述暗通道图像的清晰度小于预设清晰度，则确定车窗上存在所述第一类杂物；其中，所述暗通道图像的清晰度为所述暗通道图像中像素的灰度值的方差。
  5. 根据权利要求2-4任一项所述的车窗自动清洁方法，其特征在于，所述根据所述连续i帧车窗图像中像素的RGB值，建立图像背景模型，具体包括：
    根据所述连续i帧车窗图像中同一位置的像素的RGB值的均值，确定所述连续i帧车窗图像的均值图像；
    根据所述连续i帧车窗图像的均值图像中的像素的RGB值与第i帧车窗图像中的像素的RGB值的差值，建立所述图像背景模型。
  6. 根据权利要求2-5任一项所述的车窗自动清洁方法，其特征在于，所述根据所述图像背景模型中的像素的RGB值，确定第一区域，具体包括：
    根据所述图像背景模型中的像素的RGB值以及第一预设算法，确定第一区域； 所述第一预设算法为显著性检测算法或者目标检测算法。
  7. 根据权利要求3-6任一项所述的车窗自动清洁方法，其特征在于，若j个第二区域内同一位置的像素的RGB值未发生变化，则确定第一区域内存在第二类杂物之前，所述方法还包括：
    确定所述图像背景模型中与所述第一区域相对应的第三区域；
    确定每个第二区域中像素的RGB值与所述第三区域中同一位置的像素的RGB值的差值并进行累加，得到累加值；
    若所述累加值超过预设阈值，则确定所述j个第二区域内同一位置的像素的RGB值发生变化。
  8. 根据权利要求1至7任一项所述的车窗自动清洁方法，其特征在于，所述控制清洁工具去除杂物，具体包括：
    根据车内外温度数据控制第一清洁工具开启预设时长，去除所述第一类杂物；
    和/或
    根据行驶状态和/或驾驶员状态、所述行驶状态和/或所述驾驶员状态对应的第一预设权重参数，以及第二清洁工具连续工作的最大时长，确定第二清洁工具工作时长；
    根据所述行驶状态和/或所述驾驶员状态、所述行驶状态和/或所述驾驶员状态对应的第二预设权重参数，以及第二清洁工具工作的最大频率，确定第二清洁工具工作频率；
    根据所述第二清洁工具工作时长以及所述第二清洁工具工作频率控制第二清洁工具清洁第一区域，去除所述第二类杂物。
  9. 根据权利要求8所述的车窗自动清洁方法，其特征在于，
    根据行驶状态和/或驾驶员状态、所述行驶状态和/或所述驾驶员状态对应的第一预设权重参数，以及第二清洁工具连续工作的最大时长，确定第二清洁工具工作时长，具体包括：
    根据行驶状态、驾驶员状态、所述行驶状态对应的第一预设权重参数、所述驾驶员状态对应的第一预设权重参数、第二清洁工具连续工作的最大时长以及第二预设算法，确定第二清洁工具工作时长，所述第二预设算法为
    Figure PCTCN2020102184-appb-100001
    根据所述行驶状态和/或所述驾驶员状态、所述行驶状态和/或所述驾驶员状态对应的第二预设权重参数，以及第二清洁工具工作的最大频率，确定第二清洁工具工作频率，具体包括：
    根据所述行驶状态、所述驾驶员状态、所述行驶状态对应的第二预设权重参数、所述驾驶员状态对应的第二预设权重参数、第二清洁工具工作的最大频率以及第三预设算法，确定第二清洁工具工作频率，所述第三预设算法为
    Figure PCTCN2020102184-appb-100002
    其中，t 1为所述第二清洁工具工作时长，t s-max为所述第二清洁工具连续工作的最大时长，f为所述第二清洁工具工作频率，f max为第二清洁工具工作的最大频率，e为自然对数的底数，α 1和β 1分别为行驶状态和驾驶员状态对应的第一预设权重参数，α 1+β 1＝1，α 2和β 2分别为行驶状态和驾驶员状态对应的第二预设权重参数，α 2+β 2＝1，s 1表示行驶状态、s 2表示驾驶员状态。
  10. 根据权利要求8或9所述的车窗自动清洁方法，其特征在于，在所述根据所 述第二清洁工具工作时长以及所述第二清洁工具工作频率控制第二清洁工具清洁所述第一区域之后，所述方法还包括：
    根据所述行驶状态和/或所述驾驶员状态、所述行驶状态和/或所述驾驶员状态对应的第三预设权重参数，以及第三清洁工具连续工作的最大时长，确定第三清洁工具工作时长；
    根据所述行驶状态和/或所述驾驶员状态、所述行驶状态和/或所述驾驶员状态对应的第四预设权重参数，以及第三清洁工具工作的最大间隔，确定第三清洁工具工作间隔；
    根据所述第三清洁工具工作时长以及所述第三清洁工具工作间隔控制第三清洁工具清洁所述第一区域，并根据所述第二清洁工具工作时长以及所述第二清洁工具工作频率控制第二清洁工具清洁所述第一区域，去除所述第二类杂物。
  11. 根据权利要求10所述的车窗自动清洁方法，其特征在于，
    根据所述行驶状态和/或所述驾驶员状态、所述行驶状态和/或所述驾驶员状态对应的第三预设权重参数，以及第三清洁工具连续工作的最大时长，确定第三清洁工具工作时长，具体包括：
    根据所述行驶状态、所述驾驶员状态、所述行驶状态对应的第三预设权重参数、所述驾驶员状态对应的第三预设权重参数、第三清洁工具连续工作的最大时长以及第四预设算法，确定第三清洁工具工作时长，所述第四预设算法为
    Figure PCTCN2020102184-appb-100003
    根据所述行驶状态和/或所述驾驶员状态、所述行驶状态和/或所述驾驶员状态对应的第四预设权重参数，以及第三清洁工具工作的最大间隔，确定第三清洁工具工作间隔，具体包括：
    根据所述行驶状态、所述驾驶员状态、所述行驶状态对应的第四预设权重参数、所述驾驶员状态对应的第四预设权重参数、第三清洁工具工作的最大间隔以及第五预设算法，确定第三清洁工具工作间隔，所述第五预设算法为
    Figure PCTCN2020102184-appb-100004
    其中，t 2为所述第三清洁工具工作时长，t l-max为所述第三清洁工具连续工作的最大时长，△t为所述第三清洁工具工作间隔，△t max为第三清洁工具工作的最大间隔，e为自然对数的底数，α 3和β 3分别为所述行驶状态和所述驾驶员状态对应的第三预设权重参数，α 3+β 3＝1，α 4和β 4分别为所述行驶状态和所述驾驶员状态对应的第四预设权重参数，α 4+β 4＝1，s 1表示行驶状态、s 2表示驾驶员状态。
  12. 一种车窗自动清洁装置，其特征在于，所述装置包括：
    获取单元，用于获取车窗图像；
    确定单元，用于确定单帧车窗图像对应的暗通道图像；
    所述确定单元，还用于根据所述暗通道图像中灰度值超过预设灰度阈值的像素的数量和/或所述暗通道图像的清晰度，确定车窗上存在第一类杂物；
    和/或所述确定单元，还用于根据连续i帧车窗图像中的像素的RGB值，确定车窗上存在第二类杂物；
    控制单元，用于控制清洁工具去除杂物，所述杂物包括所述第一类杂物和/或第二类杂物。
  13. 根据权利要求12所述的车窗自动清洁装置，其特征在于，
    所述确定单元，具体用于根据所述连续i帧车窗图像中像素的RGB值，建立图像背景模型；其中，所述图像背景模型中的像素的RGB值表示所述连续i帧车窗图像中同一位置的像素的RGB值的变化，i为大于1的整数；
    根据所述图像背景模型中的像素的RGB值，确定第一区域，所述第一区域为所述连续i帧车窗图像中RGB值未发生变化的像素对应的车窗区域；
    确定所述第一区域内存在第二类杂物。
  14. 根据权利要求13所述的车窗自动清洁装置，其特征在于，
    所述确定单元，还用于在连续j帧车窗图像中的每一帧车窗图像上确定与所述第一区域相对应的一个第二区域，得到j个第二区域；所述连续j帧车窗图像位于所述连续i帧车窗图像之后，j为大于1的整数；
    若所述j个第二区域内同一位置的像素的RGB值未发生变化，则确定所述第一区域内存在第二类杂物。
  15. 根据权利要求12-14任一项所述的车窗自动清洁装置，其特征在于，
    所述确定单元，具体用于若所述暗通道图像中灰度值未超过预设灰度阈值的像素的数量小于等于所述暗通道图像中灰度值超过预设灰度阈值的像素的数量，和/或所述暗通道图像的清晰度小于预设清晰度，则确定车窗上存在所述第一类杂物；其中，所述暗通道图像的清晰度为所述暗通道图像中像素的灰度值的方差。
  16. 根据权利要求13-15任一项所述的车窗自动清洁装置，其特征在于，
    所述确定单元，具体还用于根据所述连续i帧车窗图像中同一位置的像素的RGB值的均值，确定所述连续i帧车窗图像的均值图像；
    根据所述连续i帧车窗图像的均值图像中的像素的RGB值与第i帧车窗图像中的像素的RGB值的差值，建立所述图像背景模型。
  17. 根据权利要求13-16任一项所述的车窗自动清洁装置，其特征在于，所述确定单元，具体用于根据所述图像背景模型中的像素的RGB值以及第一预设算法，确定第一区域；所述第一预设算法为显著性检测算法或者目标检测算法。
  18. 根据权利要求14-17任一项所述的车窗自动清洁装置，其特征在于，
    所述确定单元，还用于确定所述图像背景模型中与第一区域相对应的第三区域；确定每个第二区域中像素的RGB值与所述第三区域中同一位置的像素的RGB值的差值并进行累加，得到累加值；若所述累加值超过预设阈值，则确定j个第二区域内同一位置的像素的RGB值发生变化。
  19. 根据权利要求12至18任一项所述的车窗自动清洁装置，其特征在于，
    所述控制单元，具体用于根据车内外温度数据控制第一清洁工具开启预设时长，去除所述第一类杂物；
    和/或所述控制单元，具体用于根据第二清洁工具工作时长以及第二清洁工具工作频率控制第二清洁工具清洁第一区域，去除所述第二类杂物；
    其中，所述第二清洁工具工作时长根据行驶状态和/或驾驶员状态、所述行驶状态和/或所述驾驶员状态对应的第一预设权重参数、以及第二清洁工具连续工作的最大时长确定，第二清洁工具工作频率根据所述行驶状态和/或所述驾驶员状态、所述行驶状态和/或所述驾驶员状态对应的第二预设权重参数，以及第二清洁工具工作的最大频率 确定。
  20. 根据权利要求19所述的车窗自动清洁装置，其特征在于，
    所述控制单元，具体还用于根据行驶状态、驾驶员状态、所述行驶状态对应的第一预设权重参数、所述驾驶员状态对应的第一预设权重参数、第二清洁工具连续工作的最大时长以及第二预设算法，确定第二清洁工具工作时长，所述第二预设算法为
    Figure PCTCN2020102184-appb-100005
    所述控制单元，具体还用于根据所述行驶状态、所述驾驶员状态、所述行驶状态对应的第二预设权重参数、所述驾驶员状态对应的第二预设权重参数、第二清洁工具工作的最大频率以及第三预设算法，确定第二清洁工具工作频率，所述第三预设算法为
    Figure PCTCN2020102184-appb-100006
    其中，t 1为所述第二清洁工具工作时长，t s-max为所述第二清洁工具连续工作的最大时长，f为所述第二清洁工具工作频率，f max为第二清洁工具工作的最大频率，e为自然对数的底数，α 1和β 1分别为行驶状态和驾驶员状态对应的第一预设权重参数，α 1+β 1＝1，α 2和β 2分别为行驶状态和驾驶员状态对应的第二预设权重参数，α 2+β 2＝1，s 1表示行驶状态、s 2表示驾驶员状态。
  21. 根据权利要求19或20所述的车窗自动清洁装置，其特征在于，
    所述控制单元，还用于根据所述行驶状态和/或所述驾驶员状态、所述行驶状态和/或所述驾驶员状态对应的第三预设权重参数，以及第三清洁工具连续工作的最大时长，确定第三清洁工具工作时长；
    根据所述行驶状态和/或所述驾驶员状态、所述行驶状态和/或所述驾驶员状态对应的第四预设权重参数，以及第三清洁工具工作的最大间隔，确定第三清洁工具工作间隔；
    根据所述第三清洁工具工作时长以及所述第三清洁工具工作间隔控制第三清洁工具清洁所述第一区域，并根据所述第二清洁工具工作时长以及所述第二清洁工具工作频率控制第二清洁工具清洁所述第一区域，去除所述第二类杂物。
  22. 根据权利要求21所述的车窗自动清洁装置，其特征在于，
    所述控制单元，具体用于根据所述行驶状态、所述驾驶员状态、所述行驶状态对应的第三预设权重参数、所述驾驶员状态对应的第三预设权重参数、第三清洁工具连续工作的最大时长以及第四预设算法，确定第三清洁工具工作时长，所述第四预设算法为
    Figure PCTCN2020102184-appb-100007
    所述控制单元，具体用于根据所述行驶状态、所述驾驶员状态、所述行驶状态对应的第四预设权重参数、所述驾驶员状态对应的第四预设权重参数、第三清洁工具工作的最大间隔以及第五预设算法，确定第三清洁工具工作间隔，所述第五预设算法为
    Figure PCTCN2020102184-appb-100008
    其中，t 2为所述第三清洁工具工作时长，t l-max为所述第三清洁工具连续工作的最大时长，△t为所述第三清洁工具工作间隔，△t max为第三清洁工具工作的最大间隔，e为自然对数的底数，α 3和β 3分别为所述行驶状态和所述驾驶员状态对应的第三预设权重参数，α 3+β 3＝1，α 4和β 4分别为所述行驶状态和所述驾驶员状态对应的第四预设权重参数，α 4+β 4＝1，s 1表示行驶状态、s 2表示驾驶员状态。
  23. 一种车窗自动清洁装置，其特征在于，包括：处理器、存储器和通信接口；其中，通信接口用于与其他设备或通信网络通信，存储器用于存储一个或多个程序，所述一个或多个程序包括计算机执行指令，当该装置运行时，处理器执行存储器存储的所述计算机执行指令以使该装置执行如权利要求1-11任一项所述的车窗自动清洁方法。
  24. 一种计算机可读存储介质，其特征在于，包括程序和指令，当所述程序或指令在计算机上运行时，如权利要求1-11任一项所述的车窗自动清洁方法被实现。
  25. 一种包含指令的计算机程序产品，其特征在于，当所述计算机程序产品在计算机上运行时，使得所述计算机执行如权利要求1-11任一项所述的车窗自动清洁方法。
PCT/CN2020/102184 2019-08-23 2020-07-15 车窗自动清洁方法及装置 WO2021036559A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP20859291.5A EP4011715A4 (en) 2019-08-23 2020-07-15 METHOD AND DEVICE FOR AUTOMATIC CLEANING OF CAR WINDOWS
US17/678,334 US20220176912A1 (en) 2019-08-23 2022-02-23 Automatic Vehicle Window Cleaning Method and Apparatus

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910786726.5 2019-08-23
CN201910786726.5A CN112406788B (zh) 2019-08-23 2019-08-23 车窗自动清洁方法及装置

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/678,334 Continuation US20220176912A1 (en) 2019-08-23 2022-02-23 Automatic Vehicle Window Cleaning Method and Apparatus

Publications (1)

Publication Number Publication Date
WO2021036559A1 true WO2021036559A1 (zh) 2021-03-04

Family

ID=74683318

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/102184 WO2021036559A1 (zh) 2019-08-23 2020-07-15 车窗自动清洁方法及装置

Country Status (4)

Country Link
US (1) US20220176912A1 (zh)
EP (1) EP4011715A4 (zh)
CN (1) CN112406788B (zh)
WO (1) WO2021036559A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113787987A (zh) * 2021-09-13 2021-12-14 安徽江淮汽车集团股份有限公司 针对挡风玻璃前视摄像头视野区域的自清洁方法及系统
CN114604200A (zh) * 2022-03-28 2022-06-10 广州小鹏自动驾驶科技有限公司 车辆的清洁装置控制的方法、装置、车辆以及储存介质

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR102672326B1 (ko) * 2021-07-29 2024-06-05 한국전자통신연구원 무선 충전 시스템에서 영상 처리 기반 이물질 탐지 방법 및 이를 수행하는 장치
CN114881896B (zh) * 2022-07-12 2022-10-04 广东欧谱曼迪科技有限公司 内窥镜图像实时去雾方法、装置、电子设备、存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120169878A1 (en) * 2002-07-16 2012-07-05 Trw Limited Rain detection apparatus and method
CN104077750A (zh) * 2014-06-18 2014-10-01 深圳市金立通信设备有限公司 一种图像的处理方法
CN109241818A (zh) * 2017-07-11 2019-01-18 松下电器（美国）知识产权公司 附着物检测方法及装置和系统、附着物学习方法及装置、程序
CN111055811A (zh) * 2019-12-05 2020-04-24 浙江合众新能源汽车有限公司 一种基于车载智能摄像头的雨刮器控制方法及系统
CN111196122A (zh) * 2018-11-20 2020-05-26 丰田自动车株式会社 判定装置、车辆控制装置、判定方法及存储判定程序的非易失性存储介质

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE3521024C2 (de) * 1984-06-20 1994-12-08 Volkswagen Ag Scheibenreinigungsvorrichtung für Fahrzeuge
US5923027A (en) * 1997-09-16 1999-07-13 Gentex Corporation Moisture sensor and windshield fog detector using an image sensor
DE102014209197A1 (de) * 2014-05-15 2015-11-19 Conti Temic Microelectronic Gmbh Vorrichtung und Verfahren zum Erkennen von Niederschlag für ein Kraftfahrzeug
US9846927B2 (en) * 2014-05-20 2017-12-19 Qualcomm Incorporated Systems and methods for haziness detection
KR101684782B1 (ko) * 2015-05-11 2016-12-08 한국오므론전장주식회사 우적 감응형 와이퍼 장치
CN106218587B (zh) * 2016-07-25 2019-07-05 青岛海信移动通信技术股份有限公司 一种开启除雾功能的方法和设备
CN108263343A (zh) * 2016-12-31 2018-07-10 天津华方科技有限公司 应用物联网技术自动控制汽车车窗冲刷的安全装置
CN107554485B (zh) * 2017-07-18 2019-05-31 上海禹点电子科技有限公司 挡风玻璃遮挡区域擦拭控制系统及方法
CN207257290U (zh) * 2017-08-10 2018-04-20 郑州宇通客车股份有限公司 一种车辆风挡起雾识别系统及车用玻璃
CN108528395A (zh) * 2018-04-08 2018-09-14 广州大学 一种基于图像识别的车辆智能雨刮控制方法及系统

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120169878A1 (en) * 2002-07-16 2012-07-05 Trw Limited Rain detection apparatus and method
CN104077750A (zh) * 2014-06-18 2014-10-01 深圳市金立通信设备有限公司 一种图像的处理方法
CN109241818A (zh) * 2017-07-11 2019-01-18 松下电器（美国）知识产权公司 附着物检测方法及装置和系统、附着物学习方法及装置、程序
CN111196122A (zh) * 2018-11-20 2020-05-26 丰田自动车株式会社 判定装置、车辆控制装置、判定方法及存储判定程序的非易失性存储介质
CN111055811A (zh) * 2019-12-05 2020-04-24 浙江合众新能源汽车有限公司 一种基于车载智能摄像头的雨刮器控制方法及系统

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4011715A4

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113787987A (zh) * 2021-09-13 2021-12-14 安徽江淮汽车集团股份有限公司 针对挡风玻璃前视摄像头视野区域的自清洁方法及系统
CN114604200A (zh) * 2022-03-28 2022-06-10 广州小鹏自动驾驶科技有限公司 车辆的清洁装置控制的方法、装置、车辆以及储存介质
CN114604200B (zh) * 2022-03-28 2024-01-09 广州小鹏自动驾驶科技有限公司 车辆的清洁装置控制的方法、装置、车辆以及储存介质

Also Published As

Publication number Publication date
US20220176912A1 (en) 2022-06-09
EP4011715A1 (en) 2022-06-15
CN112406788A (zh) 2021-02-26
CN112406788B (zh) 2022-06-24
EP4011715A4 (en) 2023-01-11

Similar Documents

Publication Publication Date Title
WO2021036559A1 (zh) 车窗自动清洁方法及装置
US11891023B1 (en) Using camera data to manage a vehicle parked outside in cold climates
CN107209856B (zh) 环境场景状况检测
CN109476309B (zh) 高级驾驶辅助系统中的动态传感器范围
US20240070460A1 (en) Generating ground truth for machine learning from time series elements
KR102376819B1 (ko) 자체 세정 센서 하우징
WO2021259344A1 (zh) 车辆检测方法、装置、车辆和存储介质
CA3128028A1 (en) Predicting three-dimensional features for autonomous driving
CN114902070A (zh) 传感器退化检测和修复
US20170151945A1 (en) Information processing device, information processing method, control device for vehicle, and control method for vehicle
WO2021164463A1 (zh) 检测方法、装置及存储介质
CN110837800A (zh) 一种面向港口恶劣天气的目标检测和识别的方法
CN108860064A (zh) 车联网环境下前挡视线调节主动服务系统及方法
CN114932902A (zh) 一种基于车联网技术的鬼探头预警避让方法及系统
DE112020001581T5 (de) Informationsverarbeitungsvorrichtung, informationsverarbeitungsverfahren und programm
US12043309B2 (en) Vehicular control system with enhanced lane centering
CN116229407A (zh) 用于运载工具的方法、运载工具和存储介质
CN114663849B (zh) 一种两轮车转向判决及行驶环境目标检测策略
US11972613B1 (en) Apparatus and methods for atmospheric condition detection
WO2023050058A1 (zh) 控制车载摄像头的视角的方法、装置以及车辆
US20240304003A1 (en) Predicting three-dimensional features for autonomous driving
CN118082850A (zh) 一种通过路面感知控制车辆行驶的方法及系统
CN116001846A (zh) 一种面向自动驾驶的移动动作预判系统及方法
CN115083152A (zh) 一种车辆编队感知系统、方法、装置、设备及介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20859291

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2020859291

Country of ref document: EP

Effective date: 20220309