JP2018116159A - Foreign matter removal control device, foreign matter removal control method and foreign matter removal control program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technique of removing a foreign matter adhering to a camera installed in a vehicle or the like.SOLUTION: A foreign matter removal control device includes an image recognition unit which detects a foreign matter adhering to a lens of a camera from an image captured by the camera mounted on a vehicle, and a control unit which controls an air blowing unit blowing the air to the lens and a liquid injection unit injecting the liquid to the lens. The control unit performs the first removal processing of controlling the air blowing unit to blow the air or controlling the liquid injection unit to inject the liquid when the image recognition unit detects the foreign matter adhering to the lens, and performs the second removal processing different from the first removal processing and of controlling the air blowing unit to blow the air or controlling the liquid injection unit to inject the liquid when the image recognition unit detects the foreign matter adhering to the lens again after the first removal processing.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a foreign matter removal control device, a foreign matter removal control method, and a foreign matter removal control program.

  There is a system in which an image taken by a camera installed on the outside of a vehicle is displayed on a display such as a car navigation system in a vehicle room such as a passenger car so that a driver of the vehicle can check a situation outside the vehicle. In addition, there is a system that performs white line detection, stationary object detection, approaching object detection, and the like using an image taken by a camera installed outside the vehicle.

Japanese Patent Application No. 2003-209722 International Publication No. 2014/007294

  If foreign matter such as raindrops, mud, or snow adheres to the lens (surface) of the camera installed on the outside of the vehicle, it will not be possible to check the situation outside the vehicle or detect white lines. It is required to remove (also called deposits). When a foreign object adheres to the surface of the camera lens, methods for removing the foreign object from the lens surface include a camera blower (which blows compressed air onto the lens), a washer (water is applied to the lens), a camera wiper (a foreign object with a wiper) Scraping off). In most of these methods, the driver confirms the foreign matter in the image captured by the camera and performs an operation of removing the foreign matter. Since the driver of the vehicle is required to concentrate on driving the vehicle, it is desirable that the foreign matter attached to the camera is removed without any operation by the driver.

  An object of this invention is to provide the technique which removes the foreign material adhering to the camera installed in the vehicle etc.

In order to solve the above problems, the present invention employs the following means.
That is, the first aspect is
An image recognizing unit for detecting foreign matter attached to the lens of the camera from an image taken by a camera mounted on the vehicle;
An air ejection unit that ejects air to the lens, and a control unit that controls a liquid ejection unit that ejects liquid to the lens,
When the image recognition unit detects the foreign matter adhering to the lens, the control unit controls the air ejection unit to eject the air, or controls the liquid ejection unit to eject the liquid. When the image recognition unit detects the foreign matter adhering to the lens again after the first removal processing, the air ejection unit, which is different from the first removal processing, is controlled. The air is then ejected, or the liquid ejecting portion is controlled to cause the second removal process to eject the liquid.
A foreign matter removal control device is provided.
An aspect of the disclosure may be realized by executing a program by an information processing device. That is, the disclosed configuration can be specified as a program for causing the information processing apparatus to execute the processing executed by each unit in the above-described aspect, or a computer-readable recording medium on which the program is recorded. Further, the disclosed configuration may be specified by a method in which the information processing apparatus executes the process executed by each of the above-described units. The configuration of the disclosure may be specified as a system including an information processing apparatus that performs the processing executed by each of the above-described units.

  According to the present invention, foreign matter adhering to a camera installed in a vehicle or the like can be removed.

FIG. 1 is a diagram illustrating a configuration example of a system including a foreign matter removal control apparatus according to an embodiment. FIG. 2 is a diagram illustrating an arrangement example of a camera, a blower, and a washer. FIG. 3 is a diagram illustrating a hardware configuration example of the information processing apparatus. FIG. 4 is a diagram illustrating an example 1 of an operation flow for removing foreign matter in a camera installed in a vehicle. FIG. 5 is a diagram illustrating an example 2 of an operation flow for removing foreign matter in a camera installed in a vehicle. FIG. 6 is a diagram illustrating an example 3 of an operation flow for removing foreign matter in a camera installed in a vehicle.

  Hereinafter, embodiments will be described with reference to the drawings. The configuration of the embodiment is an exemplification, and the configuration of the invention is not limited to the specific configuration of the disclosed embodiment. In carrying out the invention, a specific configuration according to the embodiment may be adopted as appropriate.

<Embodiment>
(Configuration example)
FIG. 1 is a diagram illustrating a configuration example of a system including a foreign matter removal control apparatus according to the present embodiment. The system 1 shown in FIG. 1 is mounted on a vehicle such as a passenger car, and includes a foreign matter removal control device 10, an output device 20, a camera 30, a blower 40, and a washer 50. The foreign matter removal control device 10 controls the removal of foreign matter attached to the surface of the camera 30 mounted on the outside of the vehicle using the blower 40, the washer 50, and the like. Foreign objects include raindrops, mud, snow and the like. The foreign matter removal control device 10 of the present embodiment includes a control unit 11, an information acquisition unit 12, an image recognition unit 13, a blower control unit 14, and a washer control unit 15. The foreign substance removal control device 10 is connected to a vehicle shift lever 110, a radar ECU 120, a vehicle speed sensor 130, an acceleration sensor 140, and an indoor camera 150. The foreign matter removal control device 10 is connected to the output device 20, the camera 30, the blower 40, and the washer 50. The blower 40 is an example of an air ejection part. The washer 50 is an example of a liquid ejection part.

  The control unit 11 is a control unit that performs arithmetic processing for performing a foreign substance removal operation. The control unit 11 acquires information on the vehicle on which the foreign matter removal control device 10 is mounted by the information acquisition unit 12. The control unit 11 detects foreign matter attached to the lens of the camera 30 from the image captured by the camera 30 installed outside the vehicle by the image recognition unit 13. The control unit 11 uses the blower 40 and the washer 50 by the blower control unit 14 and the washer control unit 15 to remove foreign matters attached to the lens of the camera 30. The control unit 11, the blower control unit 14, and the washer control unit 15 may be integrated and operate as one control unit. Further, the control unit 11, the information acquisition unit 12, the image recognition unit 13, the blower control unit 14, and the washer control unit 15 may be integrated and operate as one control unit.

The information acquisition unit 12 acquires information such as the state of the vehicle on which the foreign substance removal control device 10 is mounted. The information acquisition unit 12 includes a shift lever 110 to a shift lever position, a radar ECU 120 to detect a target, a vehicle speed sensor 130 to a vehicle speed, an acceleration sensor 140 to a vehicle acceleration, an indoor camera 150 to a driver's line-of-sight direction, and the like. To get. The information acquired by the information acquisition unit 12 is not limited to these. The information acquisition unit 12 may acquire the temperature inside and outside the vehicle with a temperature sensor installed in the vehicle and the noise level inside and outside the vehicle with a microphone installed in the vehicle. The information acquisition unit 12 may be connected to a vehicle control unit to acquire these pieces of information. The information acquisition unit 12 stores the acquired information in a storage unit.

  The image recognition unit 13 acquires an image taken by the camera 30 installed outside the vehicle. The image recognition unit 13 detects foreign matter attached to the lens of the camera 30 from the image taken by the camera 30. The image recognition unit 13 detects foreign matter attached to the lens of the camera 30 by a known image recognition technique. Specifically, the image recognition unit 13 can detect the position of the foreign matter attached to the lens, the type of the foreign matter, and the like. For example, in the captured image, the image recognition unit 13 detects an object present at the same position for a predetermined period as a foreign matter attached to the lens. In addition, the image recognition unit 13 identifies the type of the foreign matter based on, for example, the color of the object detected as the foreign matter.

  The blower control unit 14 controls the ejection of compressed air to the blower 40 installed around the camera 30. The blower 40 can eject compressed air to the surface of the lens of the camera 30. The blower control unit 14 controls the amount and period of compressed air ejected from the blower 40.

  The washer control unit 15 controls the ejection of liquid such as water and cleaning liquid to the washer 50 installed around the camera 30. The washer 50 can eject liquid such as water or cleaning liquid onto the surface of the lens of the camera 30. The washer control unit 15 controls the amount and period of liquid such as water and cleaning liquid ejected from the washer 50.

  The output device 20 outputs an image taken by the camera 30 and the like. The output device 20 is a display, a speaker, or the like. A display of a car navigation system on which a vehicle is mounted may be used as the output device 20. A display is installed in the position which can be visually recognized from the driver's seat of a vehicle.

  The camera 30 is a photographing unit that is installed outside the vehicle and photographs a situation around the vehicle. The camera 30 includes a lens, and the surface of the lens is exposed to the outside air. Therefore, for example, when it rains around the vehicle, raindrops can adhere to the surface of the lens. A plurality of cameras 30 may be installed in the vehicle.

  The blower 40 is installed around the camera 30 and jets compressed air to the surface of the lens of the camera 30 based on an instruction from the blower control unit 14. The blower 40 includes an injection nozzle that is an injection port for compressed air. The tip of the spray nozzle is directed to the surface of the camera 30 lens. The blower 40 includes a tank that stores compressed air and a pump that injects compressed air. The pump sucks compressed air from the tank and ejects it from the injection nozzle.

  The washer 50 is installed around the camera 30 and ejects liquid such as water or cleaning liquid onto the surface of the lens of the camera 30 based on an instruction from the washer control unit 15. The washer 50 includes an injection nozzle which is a liquid injection port. The tip of the spray nozzle is directed to the surface of the camera 30 lens. The washer 50 includes a tank for storing a liquid and a pump for injecting the liquid. Liquid is sucked from the tank by the pump and ejected from the ejection nozzle.

The blower 40 and the washer 50 are installed for each camera 30. The blower 40 and the washer 50 are installed at a position that does not appear in an image captured by the camera 30. The blower 40 and the washer 50 may be integrated. Furthermore, the camera 30, the blower 40, and the washer 50 may be integrated.

  FIG. 2 is a diagram illustrating an arrangement example of a camera, a blower, and a washer. As shown in FIG. 2, the blower 40 and the washer 50 are arranged around the lens surface of the camera 30. The blow range of the blower 40 and the washer 50 includes almost the entire surface of the lens. Further, the spray nozzles of the blower 40 and the washer 50 may be movable and capable of spraying compressed air or liquid to a specific area. The blower 40 and the washer 50 may be integrated with a common injection nozzle.

  The shift lever 110 is a lever that changes the gear of the vehicle. For example, when the position of the shift lever 110 is “R (reverse)”, the vehicle moves backward. The position of the shift lever 110 is acquired by the information acquisition unit 12.

The radar ECU 120 is an electronic control unit (Electric Control Unit) that detects a target around the vehicle and calculates a distance between the target around the vehicle and the vehicle. Radar ECU1
20 calculates the distance to the target using a microwave, an image taken by the camera 30 or the like. If a foreign object adheres to the camera 30, there is a possibility that the target detection or the like cannot be performed accurately.

  The vehicle speed sensor 130 is a sensor that detects the traveling speed of the vehicle. The vehicle speed sensor 130 detects and outputs the vehicle speed based on, for example, the tire rotation angle and the tire radius.

  The acceleration sensor 140 is a sensor that detects and outputs the acceleration of the vehicle.

  The indoor camera 150 is a camera that captures the interior of the vehicle. For example, the indoor camera 150 detects the line-of-sight direction of the driver of the vehicle.

  FIG. 3 is a diagram illustrating a hardware configuration example of the information processing apparatus. The information processing apparatus 90 shown in FIG. 3 has a general computer configuration. The foreign matter removal control device 10 is realized by using an information processing device 90 as shown in FIG. 3 includes a processor 91, a memory 92, a storage unit 93, an input unit 94, an output unit 95, and a communication control unit 96. These are connected to each other by a bus. The memory 92 and the storage unit 93 are computer-readable recording media. The hardware configuration of the information processing apparatus is not limited to the example illustrated in FIG. 3, and omissions, replacements, and additions of components may be appropriately performed.

  In the information processing apparatus 90, a processor 91 loads a program stored in a recording medium into a work area of the memory 92 and executes the program, and each component is controlled through execution of the program, thereby meeting a predetermined purpose. Function can be realized.

  The processor 91 is, for example, a CPU (Central Processing Unit) or a DSP (Digital Signal Processor).

  The memory 92 includes, for example, a RAM (Random Access Memory) and a ROM (Read Only Memory). The memory 92 is also called a main storage device.

The storage unit 93 is, for example, an EPROM (Erasable Programmable ROM) or a hard disk drive (HDD, Hard Disk Drive). The storage unit 93 can include a removable medium, that is, a portable recording medium. Removable media is, for example, USB (Universal Serial Bus) memory, CD (Compact Disc) or DVD (Digital
Versatile Disc). The storage unit 93 is also called a secondary storage device.

  The storage unit 93 stores various programs, various data, and various tables in a recording medium in a readable and writable manner. The storage unit 93 stores an operating system (OS), various programs, various tables, and the like. Information stored in the storage unit 93 may be stored in the memory 92. In addition, information stored in the memory 92 may be stored in the storage unit 93.

  The operating system is software that mediates software and hardware, manages memory space, manages files, manages processes and tasks, and the like. The operating system includes a communication interface. The communication interface is a program for exchanging data with other external devices connected via the communication control unit 96. Examples of the external device include other information processing devices and external storage devices.

  The input unit 94 includes a keyboard, a pointing device, a wireless remote controller, a touch panel, and the like. The input unit 94 may include a video / image input device such as a camera, and an audio input device such as a microphone.

The output unit 95 includes an LCD (Liquid Crystal Display), an EL (Electroluminescence) panel, a CRT (Cathode Ray Tube) display, a display device such as a PDP (Plasma Display Panel), and an output device such as a printer. The output unit 95 can include an audio output device such as a speaker.

  The communication control unit 96 is connected to another device and controls communication between the information processing device 90 and the other device. The communication control unit 96 is, for example, a LAN (Local Area Network) interface board, a wireless communication circuit for wireless communication, or a communication circuit for wired communication. The LAN interface board and the wireless communication circuit are connected to a network such as the Internet.

(Operation example 1)
FIG. 4 is a diagram illustrating an example 1 of an operation flow for removing foreign matter in a camera installed in a vehicle. The operation flow of FIG. 4 is an operation flow executed when electric power is supplied to the foreign matter removal control apparatus 10 from a vehicle on which the foreign matter removal control apparatus 10 is mounted. When the vehicle drive source is in an operating state, for example, when the drive source is an internal combustion engine, the ignition switch is turned on, or when the system power source is turned on in a hybrid system or EV (Electric Vehicle) system, FIG. The operation flow is started.

  In S101, the control unit 11 of the foreign matter removal control apparatus 10 determines whether or not the vehicle information acquired by the information acquisition unit 12 is in a predetermined state. The predetermined state is, for example, a case where the shift lever 110 is in the “R (reverse)” position, a case where a predetermined object is detected by the radar ECU 120, a case where the vehicle starts, or another important case. An important case is, for example, a case where the vehicle may convey some information to the driver using an image captured by the camera 30. If the vehicle information is in a predetermined state (S101; YES), the process proceeds to S102. If the vehicle information is not in a predetermined state (S101; NO), the process repeats S101. This is because it is not necessary to remove the foreign matter when it is not in the predetermined state.

In S <b> 102, the control unit 11 of the foreign matter removal control apparatus 10 instructs the image recognition unit 13 to capture an image with the camera 30. The image recognition unit 13 takes an image with the camera 30. The image captured by the camera 30 may be a still image or a moving image. The image to be captured may be a plurality of still images. The image recognition unit 13 determines whether or not there is a foreign object attached to the lens in the image captured by the camera 30. The image recognition unit 13 determines the presence of a foreign object, for example, using a known image recognition technique. When a foreign object exists in the lens of the camera 30 (S102; YES), the process proceeds to S103. If there is no foreign object on the lens of the camera 30 (S102; NO), the process ends.

  In S103, the control unit 11 of the foreign matter removal control apparatus 10 instructs the blower control unit 14 to eject the first predetermined amount of compressed air to the blower 40 for the first predetermined period as the first removal operation. . The blower control unit 14 causes the blower 40 to eject a first predetermined amount of compressed air for a first predetermined period. The blower 40 ejects a first predetermined amount of compressed air toward the camera 30 for a first predetermined period.

  In S <b> 104, the control unit 11 instructs the image recognition unit 13 to capture an image again with the camera 30. The image recognition unit 13 takes an image with the camera 30. The image captured by the camera 30 may be a still image or a moving image. The image to be captured may be a plurality of still images. The image recognition unit 13 determines whether or not there is a foreign object attached to the lens in the image captured by the camera 30. That is, it is determined whether or not the foreign matter detected in S102 has been removed by the first removal operation. When a foreign object exists in the lens of the camera 30 (S104; YES), the process proceeds to S105. If there is no foreign object on the lens of the camera 30 (S104; NO), the process ends. The foreign matter detected here may be the same type of foreign matter as the foreign matter detected in S102 and the same type of foreign matter. This is because when the foreign matter detected in S102 is not removed in the first removal operation in S103, the second removal operation in S105 is performed.

  In S105, the control unit 11 instructs the blower control unit 14 to eject the second predetermined amount of compressed air to the blower 40 for the second predetermined period as the second removal operation. Here, the second predetermined amount is larger than the first predetermined amount, and the second predetermined period is longer than the first predetermined period. The blower control unit 14 causes the blower 40 to eject a second predetermined amount of compressed air for a second predetermined period. The blower 40 ejects a second predetermined amount of compressed air toward the camera 30 for a second predetermined period. The second predetermined period may be equal to or shorter than the first predetermined period. By ejecting more air in a shorter time, the pressure of the compressed air to be ejected can be increased. By the second removal operation, the blower control unit 14 can cause the lens of the camera 30 to eject compressed air stronger than the first removal operation.

  In step S <b> 106, the control unit 11 instructs the image recognition unit 13 to capture an image again with the camera 30. The image recognition unit 13 takes an image with the camera 30. The image captured by the camera 30 may be a still image or a moving image. The image to be captured may be a plurality of still images. The image recognition unit 13 determines whether or not there is a foreign object attached to the lens in the image captured by the camera 30. That is, it is determined whether or not the foreign matter detected in S104 has been removed by the second removal operation. If there is a foreign object on the lens of the camera 30 (S106; YES), the process proceeds to S107. If there is no foreign object on the lens of the camera 30 (S106; NO), the process ends. The foreign matter detected here may be a foreign matter of the same position as the foreign matter detected in S104 and the same kind of foreign matter. This is because the third removal operation of S107 is performed when the foreign matter detected in S104 is not removed by the second removal operation of S105.

In S107, as a third removal operation, the control unit 11 instructs the washer control unit 15 to eject a third predetermined amount of liquid (water or cleaning liquid) to the washer 50 for a third predetermined period. The washer control unit 15 causes the washer 50 to eject a third predetermined amount of liquid for a third predetermined period. The washer 50 ejects a third predetermined amount of liquid toward the camera 30 during the third predetermined period. As a result, the washer control unit 15 can cause the lens of the camera 30 to perform a stronger removal operation than the first removal operation and the second removal operation.

  Thereby, a series of removal operations are completed. The foreign matter removal control apparatus 10 performs three types of removal operations in accordance with the foreign matter removal status in order to remove foreign matter attached to the lens of the camera 30. After the end, the process may return to S101 again. Instead of the compressed air being blown out by the blower 40, the liquid may be blown out by the washer 50, or instead of the liquid being jetted out by the washer 50, the compressed air may be blown out by the blower 40.

(Modification 1)
As the second removal operation or the third removal operation, the control unit 11 ejects a fourth predetermined amount of liquid to the washer 50 through the washer control unit 15 for a fourth predetermined period, and then the fifth predetermined operation. After the period has elapsed, a sixth predetermined amount of compressed air may be ejected to the blower 40 through the blower control unit 14 for a sixth predetermined period. Thereby, foreign matter removal can be performed more reliably.

  After the third removal operation in S107, the control unit 11 again determines whether or not there is a foreign object. If the foreign object still exists, the control unit 11 displays a display indicating that the foreign object has not been removed. You may let them. In addition, the control unit 11 may cause the output device 20 to output a voice or sound indicating that the foreign matter has not been removed. As a result, the vehicle driver recognizes that the foreign object could not be removed by the foreign object removal operation, and stops the vehicle at a safe position, for example, and attaches to the camera 30 by himself (with a towel or the like). Foreign matter can be removed.

  When the blower 40 or the washer 50 is not present in the vehicle and the foreign object is detected by the process of S102 or the like, the control unit 11 causes the output device 20 to display that there is a foreign object on the lens of the camera 30. Good. Further, the control unit 11 may cause the output device 20 to output a sound or sound indicating that a foreign object exists. Thereby, the driver of the vehicle recognizes that a foreign object exists in the lens of the camera 30 even if the blower 40 and the washer 50 are not present in the vehicle, and stops the vehicle at a safe position. The foreign matter attached to the camera 30 can be removed by itself (with a towel or the like). The notification of the presence of the foreign matter may be performed not only when the foreign matter is attached but also when the foreign matter is present for a predetermined time or more. In addition, the notification of the presence of a foreign object may be given a stronger alert (a conspicuous warning display, a large volume, etc.) when there is no background image at the location where the foreign object is attached (the spatial frequency is low).

  The controller 11 may cause the blower 40 to inject the compressed air a predetermined number of times or the washer 50 to inject the liquid a predetermined number of times as the first removal operation in S103. Further, the control unit 11 may determine the presence of foreign matter in S104 after a predetermined time of the first removal operation. At this time, if the foreign object still exists, the output device 20 may output a display indicating that the foreign object is present on the lens of the camera 30 or an audio output. Accordingly, the durability of the blower 40 and the washer 50 can be ensured by reducing the number of operations of the blower 40 and the washer 50.

  In each removal operation, the control unit 11 may change the ejection amount, the period, the number of times, and the ejection pressure of the compressed air or liquid depending on the type of foreign matter. For example, in the case of raindrops in which foreign matter is easily removed, the ejection amount, period, number of times, and ejection pressure are reduced, and in the case of mud in which foreign matter is difficult to remove, the ejection amount, period, number of times, and ejection pressure are increased.

In each removal operation, the control unit 11 may change the amount, period, number, and ejection pressure of compressed air or liquid depending on the application of the camera 30. For example, when a foreign object adheres to a camera 30 that requires high reliability (for example, a camera used for sensing), the ejection amount, the period, the number of times, and the ejection pressure are increased. This makes it easier to remove foreign matter from the camera 30 that requires high reliability.

  The control part 11 is good also as a case where the driver | operator's eyes | visual_axis direction is the direction of the display (such as a car navigation system) in a vehicle in operation | movement of S101. In other words, the control unit 11 determines that the driver's line-of-sight direction acquired by the information acquisition unit 12 from the indoor camera 150 is a predetermined state when the direction of the display is the display direction. This is because when the driver's line-of-sight direction is the direction of the display and the foreign object is attached to the lens of the camera 30, the driver cannot recognize the image taken by the camera 30.

  While the driver's line-of-sight direction is the direction of the display in the vehicle, the control unit 11 performs the determination of the presence of a foreign object (S102, etc.) and the foreign object removal operation (S103, etc.) when a foreign object is attached. It may be repeated.

  Moreover, the control part 11 is good also as a case where a driver | operator's gaze direction is not the direction of the display (such as a car navigation system) in a vehicle in the operation | movement of S101. That is, the control unit 11 determines that the driver is in a predetermined state when the driver's line-of-sight direction acquired by the information acquisition unit 12 from the indoor camera 150 is not the display direction. It is not necessary for the driver to show the lens 30 foreign object removal operation. When the driver's line-of-sight direction is the display direction, the driver sees the foreign object removal operation when the camera 30 lens foreign object removal operation is performed. Because it will be.

  The control unit 11 may acquire information on the driver's line-of-sight direction at predetermined time intervals. At this time, the control unit 11 may determine whether to detect a foreign object (S101; YES) or not (S101; NO) based on the driver's last line of sight information acquired. As a result, the foreign matter removing operation can be continuously performed without continuously acquiring information on the driver's line-of-sight direction.

  When a plurality of cameras 30 are present in the vehicle, the control unit 11 may perform a foreign matter removing operation in any camera when any one of the cameras 30 detects the foreign matter. This is because when a foreign object is detected by one camera 30, there is a possibility that the other camera 30 may have a foreign object attached thereto. In addition, an operation sound is generated in the foreign matter removing operation by the blower 40 or the washer 50. When the foreign substance removing operation is performed simultaneously by the plurality of cameras 30, there is a possibility that a larger operating sound is generated. Therefore, the timing of the foreign substance removal operation in each camera 30 can be shifted so as not to overlap. The operation sound of the foreign substance removal operation by the camera 30 can be suppressed.

(Operation example 2)
FIG. 5 is a diagram illustrating an example 2 of an operation flow for removing foreign matter in a camera installed in a vehicle. The operation flow of FIG. 5 is an operation flow executed when power is supplied to the foreign matter removal control apparatus 10 from a vehicle on which the foreign matter removal control apparatus 10 is mounted. When the driving source of the vehicle is in an operating state, for example, when the ignition source is turned on if the driving source is an internal combustion engine, or when the system power is turned on for a hybrid system or EV system, the operation flow of FIG. Be started. Here, the foreign substance removal operation is changed based on the position, size, and type of the foreign substance.

  In S201, the control unit 11 of the foreign matter removal control apparatus 10 determines whether or not the vehicle information acquired by the information acquisition unit 12 is in a predetermined state. The operation in S201 is the same as the operation in S101 in FIG.

In S <b> 202, the control unit 11 of the foreign matter removal control apparatus 10 determines whether there is a foreign matter attached to the lens of the camera 30 via the image recognition unit 13. The operation of S202 is the same as the operation of S102 of FIG.

  In S203, the image recognition unit 13 of the foreign matter removal control apparatus 10 detects the position, size, and type of the foreign matter detected in S102 in the image taken by the camera 30. The image recognition unit 13 detects at least one of the position, size, and type of the foreign object. The image recognizing unit 13 can detect the position, size, and type of a foreign object by a known image recognition method. The position of the foreign matter in the image is represented by coordinates in the foreign matter image, for example. Further, since the positions of the ejection nozzles of the blower 40 and the washer 50 are known, the distance between the foreign objects and the ejection nozzle can be easily obtained from the coordinates of the foreign objects and the position of the ejection nozzle. The size of the foreign matter is represented by the ratio of pixels that are foreign matter in the image, the number of pixels, the area of foreign matter, and the like. The type of foreign matter can be determined by a known image recognition method. For example, the type of foreign matter is determined by the color of the foreign matter. The types of foreign matter include raindrops, mud, snow, and the like.

  In S204, the control unit 11 of the foreign matter removal control device 10 compresses the blower 40 with a predetermined amount, a predetermined period, a predetermined number of times, and a predetermined ejection pressure with respect to the blower control unit 14 or the washer control unit 15 as a foreign matter removal operation. The air or washer 50 is instructed to eject liquid. The control unit 11 determines the predetermined amount, the predetermined period, the predetermined number of times, and the predetermined ejection pressure based on at least one of the distance between the foreign object and the ejection nozzle, the size of the foreign object, and the type of the foreign object. As the distance between the foreign object and the ejection nozzle is longer, the control unit 11 increases the ejection amount, the period, the number of times, and the ejection pressure. The control unit 11 increases the amount, the period, the number of times, and the ejection pressure to be ejected as the size of the foreign matter is larger. The control unit 11 increases the amount, the period, the number of times, and the ejection pressure to be ejected as the type of foreign matter is more likely to stick to the lens. For example, when the type of the foreign matter is raindrops, the control unit 11 determines that the sticking is difficult, and when the foreign matter is mud, the control unit 11 determines that the sticking is easy.

  Thereby, a series of removal operations are completed. The foreign matter removal control device 10 performs a removal operation according to the state of the foreign matter in order to remove the foreign matter attached to the lens of the camera 30. After the end, the process may return to S201 again. Thereby, the foreign substance removal operation | movement according to the condition of a foreign substance can be performed.

(Modification 2)
The control unit 11 may gradually decrease the ejection amount as the number of ejections of compressed air or liquid increases in the foreign substance removal operation in S204. At this time, the amount of compressed air or liquid ejected can be saved.

  The controller 11 may repeat the foreign object detection (S202) and the foreign object removal operation (S204) a plurality of times as in the example of FIG. At this time, the controller 11 may detect the position, size, and type of the foreign object (S203) after the foreign object detection (S202). Further, when the size of the foreign matter detected at the second time is 10% or more of the size of the foreign matter detected at the first time, the control unit 11 performs the second removal operation and detects the third time. The third removal operation may be performed when the size of the foreign matter detected is 5% or more of the size of the foreign matter detected the first time. Thus, when more than 90% of the foreign matter detected at the first time is removed, the second and subsequent removal operations are not performed, and when more than 95% of the foreign matter detected at the first time is removed, 3 is removed. The second removal operation is not performed.

In addition, when the position of the foreign matter detected for the second time is within the first distance from the center of the image, the control unit 11 performs the second removal operation and determines the position of the foreign matter detected for the third time. The third removal operation may be performed when the distance is within the second distance shorter than the first distance from the center of the image. As a result, when there is no foreign object within the first distance from the center of the image, the second and subsequent removal operations are not performed, and when there is no foreign object within the second distance from the center of the image, the third time. The removal operation is not performed. Since the central portion of the image is considered to be more important than the peripheral portion of the image, the foreign matter removing operation is repeatedly performed when the foreign matter in the central portion of the image is not removed. As a result, the foreign matter removing operation is performed with priority given to the central portion of the image.

  The control unit 11 may reduce the ejection amount in the foreign matter removing operation when the temperature outside the vehicle is a predetermined value or less (for example, 0 ° C. or less). Moreover, the control part 11 may reduce the ejection amount in a foreign material removal operation, when the detected foreign material is predetermined snow (for example, smooth snow). This is because compressed air or liquid may freeze. The smooth snow is, for example, snow that falls at a temperature of −5 ° C. or lower. The temperature (air temperature) outside the vehicle can be acquired by a temperature sensor installed in advance in the vehicle.

(Operation example 3)
FIG. 6 is a diagram illustrating an example 3 of an operation flow for removing foreign matter in a camera installed in a vehicle. The operation flow of FIG. 6 is an operation flow executed when electric power is supplied to the foreign matter removal control apparatus 10 from a vehicle on which the foreign matter removal control apparatus 10 is mounted. When the driving source of the vehicle is in an operating state, for example, when the ignition source is turned on if the driving source is an internal combustion engine, or when the system power is turned on if it is a hybrid system or EV system, the operation flow of FIG. Be started. Here, the foreign matter removing operation is performed when another sound is generated.

  In S301, the control unit 11 of the foreign matter removal control apparatus 10 determines whether or not the vehicle information acquired by the information acquisition unit 12 is in a predetermined state. The operation in S301 is the same as the operation in S101 in FIG.

  In step S <b> 302, the control unit 11 of the foreign matter removal control apparatus 10 determines whether there is foreign matter that adheres to the lens of the camera 30 via the image recognition unit 13. The operation of S302 is the same as the operation of S102 of FIG.

  In S <b> 303, the control unit 11 determines whether or not an operation for generating a sound is performed in the vehicle information acquired by the information acquisition unit 12. If an operation that generates sound is performed (S303; YES), the process proceeds to S304. When the operation for generating sound is not performed (S303; NO), the process of S303 is repeated. The operation that generates the sound is, for example, an operation in which the position of the shift lever 110 in which the back warning sound is generated is set to “R (reverse)” or an operation in which a horn button that generates the horn sound is pressed. The operation in which sound is generated may include a case where the noise level outside or inside the vehicle is a predetermined value or more. The noise level is acquired via the information acquisition unit 12 by input means such as a microphone installed outside the vehicle or inside the vehicle.

  In S304, the control unit 11 of the foreign matter removal control apparatus 10 compresses the blower 40 with a predetermined amount, a predetermined period, a predetermined number of times, and a predetermined ejection pressure with respect to the blower control unit 14 or the washer control unit 15 as a foreign matter removal operation. The air or washer 50 is instructed to eject liquid. The foreign matter removing operation is performed in accordance with the operation of generating the sound in S303. By performing according to the operation in which the sound is generated, the sound generated in the foreign matter removing operation can be made unconscious. The control unit 11 of the foreign matter removal control apparatus 10 may make the amount of compressed air or liquid ejected larger than usual.

  Thereby, the foreign material removal control apparatus 10 can mix the sound generated by the foreign material removal operation with the sound generated by other operations.

(Other)
The foreign matter removal control device 10 is one of the predetermined amount, period, number of times, and pressure of the air or liquid to be ejected when a predetermined operation is performed by a manual operation, that is, a vehicle driver or the like. The removal process may be performed under the control. The foreign matter removal control device 10 may perform the removal process when a manual operation is performed by a vehicle driver and foreign matter attached to the lens is detected by image recognition.

  When the manual operation is continuously performed within a predetermined period, the foreign matter removal control apparatus 10 may change the predetermined amount, period, number, and pressure of air or liquid to be ejected according to the number of times. For example, as the number of manual operations increases, a predetermined amount, period, number, and pressure of air or liquid to be ejected are increased. In addition, the foreign matter removal control device 10 performs the first automatic operation for the first manual operation, that is, the manual operation after a predetermined period of time has passed since the previous manual operation, and the predetermined air or liquid to be ejected. The amount, period, number of times, and pressure may be the same.

  As mentioned above, although embodiment of this invention was described, these are only illustrations to the last, this invention is not limited to these, Based on the knowledge of those skilled in the art, unless it deviates from the meaning of a claim Various changes are possible. Moreover, each structural example etc. may be combined and implemented as much as possible.

<Computer-readable recording medium>
A program for causing a computer or other machine or device (hereinafter, a computer or the like) to realize any of the above functions can be recorded on a recording medium that can be read by the computer or the like. The function can be provided by causing a computer or the like to read and execute the program of the recording medium.

  Here, a computer-readable recording medium is a recording medium that stores information such as data and programs by electrical, magnetic, optical, mechanical, or chemical action and can be read from a computer or the like. Say. In such a recording medium, elements constituting a computer such as a CPU and a memory may be provided to cause the CPU to execute a program.

  Examples of such a recording medium that can be removed from a computer or the like include a flexible disk, a magneto-optical disk, a CD-ROM, a CD-R / W, a DVD, a DAT, an 8 mm tape, and a memory card.

  Moreover, there are a hard disk, a ROM, and the like as a recording medium fixed to a computer or the like.

DESCRIPTION OF SYMBOLS 1 System 10 Foreign substance removal control apparatus 11 Control part 12 Information acquisition part 13 Image recognition part 14 Blower control part 15 Washer control part 20 Output device 30 Camera 40 Blower 50 Washer 110 Shift lever 120 Radar ECU
130 Vehicle speed sensor 140 Acceleration sensor 150 Indoor camera

Claims (9)

An image recognizing unit for detecting foreign matter attached to the lens of the camera from an image taken by a camera mounted on the vehicle;
An air ejection unit that ejects air to the lens, and a control unit that controls a liquid ejection unit that ejects liquid to the lens,
When the image recognition unit detects the foreign matter adhering to the lens, the control unit controls the air ejection unit to eject the air, or controls the liquid ejection unit to eject the liquid. When the image recognition unit detects the foreign matter adhering to the lens again after the first removal processing, the air ejection unit, which is different from the first removal processing, is controlled. The second removal process for ejecting the air or controlling the liquid ejection part to eject the liquid is performed.
Foreign matter removal control device. The control unit outputs information to a driver of the vehicle when the image recognition unit detects the foreign matter adhering to the lens again after a lapse of a predetermined period after performing the first removal process. Outputs to the device that the foreign matter is attached to the lens.
The foreign matter removal control apparatus according to claim 1. When the image recognition unit detects the foreign matter adhering to the lens after the second removal process, the control unit outputs the information to a driver of the vehicle. Output that it is attached,
The foreign matter removal control apparatus according to claim 1. The control unit changes the content of the first removal process or the second removal process according to the use of the camera.
The foreign matter removal control apparatus according to any one of claims 1 to 3. The control unit causes the first removal process and the second removal process to be performed when the line-of-sight direction of the driver of the vehicle is a direction of a display mounted on the vehicle.
The foreign matter removal control apparatus according to any one of claims 1 to 4. The controller causes the first removal process and the second removal process to be performed when the line-of-sight direction of the driver of the vehicle is not the direction of the display mounted on the vehicle.
The foreign matter removal control apparatus according to any one of claims 1 to 4. The control unit causes the first removal process and the second removal process to be continuously performed when the line-of-sight direction of the driver of the vehicle is a direction of a display mounted on the vehicle.
The foreign matter removal control apparatus according to any one of claims 1 to 4. Computer
Detect foreign matter adhering to the camera lens from the image taken by the camera mounted on the vehicle,
Controlling an air ejection part for ejecting air to the lens and a liquid ejection part for ejecting liquid to the lens;
When the foreign matter adhering to the lens is detected, the air ejection unit is controlled to eject the air, or the liquid ejection unit is controlled to perform a first removal process for ejecting the liquid, When the foreign matter adhering to the lens is detected again after the first removal process, the air ejection part is controlled to eject the air, or the liquid ejection part is different from the first removal process. Performing a second removal process for controlling and ejecting the liquid;
Foreign matter removal control method for executing the above. Computer
Detect foreign matter adhering to the camera lens from the image taken by the camera mounted on the vehicle,
Controlling an air ejection part for ejecting air to the lens and a liquid ejection part for ejecting liquid to the lens;
When the foreign matter adhering to the lens is detected, the air ejection unit is controlled to eject the air, or the liquid ejection unit is controlled to perform a first removal process for ejecting the liquid, When the foreign matter adhering to the lens is detected again after the first removal process, the air ejection part is controlled to eject the air, or the liquid ejection part is different from the first removal process. Performing a second removal process for controlling and ejecting the liquid;
Foreign matter removal control program for executing this.

Images