JP2022182057A - Illuminance estimation device - Google Patents

Abstract

To provide an illuminance estimation device which can improve estimation accuracy of illuminance.SOLUTION: An illuminance estimation device 100 comprises: cameras 101-110; a navigation system 112 or communication module 113; and an electronic control unit 115. The cameras 101-110 are mounted on an own vehicle 111 and image a region which is directly or indirectly irradiated with light of the sun 200. The navigation system 112 or the communication module 113 acquires vehicle information including at least the current position of the own vehicle 111, the current orientation of the own vehicle 111 and the current date and time. The electronic control unit 115 acquires images from the cameras 101-110 and acquires the vehicle information from the navigation system 112 or the communication module 113. The electronic control unit 115 estimates the current position of the sun 200 and illuminance around the own vehicle 111 on the basis of the images and the vehicle information.SELECTED DRAWING: Figure 1

Description

The present invention relates to an illuminance estimation device.

2. Description of the Related Art Conventionally, a device for estimating the illuminance around a vehicle using an image taken in front of the vehicle has been proposed, for example, in Japanese Unexamined Patent Application Publication No. 2002-100000. The device has, as learning data, a learning image captured in advance, the time when the learning image was captured, and the illuminance measured around the vehicle at the time when the learning image was captured. That is, the device estimates the illuminance around the vehicle at the time when the image was captured based on the learning data based on the captured image and the time when the image was captured.

JP 2020-175743 A

However, in the conventional technology described above, since only an image of the area in front of the vehicle is used as information about the surroundings of the vehicle, the information for estimating the illuminance is biased toward the situation in front of the vehicle. Therefore, there is a difference between the actual illuminance and the estimated illuminance based on the road surface condition, weather, sun position, sun height, and the like. As a result, the accuracy of illuminance estimation decreases.

SUMMARY OF THE INVENTION An object of the present invention is to provide an illuminance estimation device capable of improving the accuracy of illuminance estimation.

To achieve the above object, in the first aspect of the invention, the illuminance estimation device includes cameras (101 to 110), communication units (112, 113), and an estimation unit (115).

The camera is mounted on the vehicle (111) and captures an area illuminated directly or indirectly by the sun (200). The communication unit acquires vehicle information including at least the current position of the vehicle, the current orientation of the vehicle, and the current date and time.

The estimation unit acquires an image from the camera, acquires vehicle information from the communication unit, and estimates the current position of the sun and the illuminance around the vehicle based on the image and the vehicle information.

According to this, since the illuminance is estimated using the vehicle information and the image containing a plurality of pieces of information, the information for estimating the illuminance is not biased toward the situation ahead of the own vehicle. Therefore, the difference between the actual illuminance and the estimated illuminance can be reduced. Therefore, the accuracy of illuminance estimation can be improved.

It should be noted that the reference numerals in parentheses of each means described in this column and claims indicate the correspondence with specific means described in the embodiments described later.

1 is a block diagram including an illuminance estimation device according to a first embodiment; FIG. FIG. 3 is a schematic diagram showing installation locations of respective cameras in the own vehicle; It is the figure which showed an example of the operation|movement of an illuminance estimation apparatus. It is the figure which showed an example of the direction of the present sun with respect to the own vehicle.

An embodiment of the present invention will be described below with reference to the drawings. In addition, in each of the following embodiments, the same or equivalent portions are denoted by the same reference numerals in the drawings.

(First embodiment)
A first embodiment will be described below with reference to the drawings. The illuminance estimation device according to this embodiment is mounted on a vehicle. The illuminance estimating device is a device that estimates the illuminance around the vehicle and determines control of turning on/off the lights and control of the air conditioner based on the estimated illuminance.

As shown in FIGS. 1 and 2, the illuminance estimation device 100 includes a driver detection camera 101, an occupant detection camera 102, a rain camera 103, a front camera 104, a plurality of peripheral cameras 105 to 108, an electronic mirror camera 109, 110 included. Each of the cameras 101 to 110 is mounted on a vehicle 111 and is a photographing device for photographing an area directly or indirectly illuminated by sunlight.

The illuminance estimation device 100 also includes a navigation system 112 , a communication module 113 , a temperature sensor 114 and an electronic control unit 115 .

The driver detection camera 101 photographs the driver of the own vehicle 111 . The driver detection camera 101 is, for example, a part of a DSM (Driver Status Monitor) for monitoring the driver's doze, distraction, and driving posture by photographing the driver's face. The DSM includes, for example, a near-infrared light source, a driver detection camera 101, a control unit that controls them, and the like.

The driver detection camera 101 is installed, for example, on the upper surface of the steering column, the upper surface of the instrument panel, or the like so as to photograph the headrest portion of the driver's seat. The driver detection camera 101 captures an image of the driver's head irradiated with near-infrared light from a near-infrared light source.

The DSM's control unit analyzes the driver image captured by the driver detection camera 101 . The control unit extracts information such as the position of the driver's right eye, the position of the left eye, etc. from the driver image, and issues a voice message to the driver based on the extracted eyeball position information. In addition, the DSM control unit outputs driver image data to the electronic control unit 115 in response to a request from the electronic control unit 115 .

The occupant detection camera 102 photographs the occupants inside the vehicle 111 . The occupant detection camera 102 is, for example, part of an occupant detection system that detects the state of occupants in the interior of the vehicle 111 . The occupant detection camera 102 is installed, for example, on the back side of the room mirror or on the ceiling.

The occupant includes both the case where the driver is included and the case where the driver is not included. When the occupant detection camera 102 captures the rear seats, the driver is not included in the occupant image captured by the occupant detection camera 102 . When the occupant detection camera 102 captures images including the driver's seat and the front passenger's seat, the driver is included in the occupant image. The occupant detection camera 102 outputs occupant image data to the electronic control unit 115 in response to a request from the electronic control unit 115 .

The rain camera 103 is a rain sensor camera that captures raindrops adhering to the windshield 116 of the vehicle 111 . The rain camera 103 is installed, for example, behind the rearview mirror or on the windshield 116 . The rain camera 103 is part of a raindrop detection device that controls the wiper device of the own vehicle 111 according to the amount of raindrops adhering to the windshield 116 . The rain camera 103 outputs rain image data to the electronic control unit 115 in response to a request from the electronic control unit 115 .

The front camera 104 captures an image of the front of the vehicle 111 with respect to the traveling direction of the vehicle 111 . Front camera 104, for example, together with electronic control unit 115 constitutes an Advanced Driver Assistance System (ADAS). Front camera 104 outputs forward image data to electronic control unit 115 in response to a request from electronic control unit 115 .

A plurality of peripheral cameras 105 to 108 capture images of the surroundings of the vehicle 111 in order to grasp the position of the vehicle 111 in a bird's-eye view with respect to the traveling direction of the vehicle 111 . Peripheral camera 105 photographs the front of own vehicle 111 . The peripheral camera 105 is installed on the front part of the own vehicle 111 . Peripheral camera 106 captures the rear of own vehicle 111 . The peripheral camera 106 is installed in the rear portion of the own vehicle 111 . Peripheral camera 107 captures the left side of own vehicle 111 . The peripheral camera 107 is installed on the left side of the own vehicle 111 such as an electronic mirror camera 109 or a front passenger side door. Peripheral camera 108 captures the right side of own vehicle 111 . The peripheral camera 108 is installed on the right side of the own vehicle 111 such as the electronic mirror camera 110 or the driver's side door.

The peripheral cameras 105 to 108 are part of a peripheral monitoring system for assisting the driver's field of vision when, for example, the vehicle 111 is parked, stopped, or started. The surroundings monitoring system acquires each data of each surrounding image from each of the surrounding cameras 105 to 108 and performs image processing such as synthesizing each surrounding image. Then, the perimeter monitoring system displays a bird's-eye view image including the surrounding scenery on the display of the navigation system 112 or the display of the meter. Peripheral cameras 105 to 108 also output each data of each peripheral image to electronic control unit 115 in response to a request from electronic control unit 115 .

Electronic mirror cameras 109 and 110 photograph the rear and rear sides of the vehicle 111 . The electronic mirror cameras 109 and 110 are part of an electronic mirror system that displays, for example, images of the rear and rear sides of the own vehicle 111 on indoor door mirror displays. The electronic mirror camera 109 is installed at a position corresponding to the left door mirror of the own vehicle 111 . The electronic mirror camera 110 is installed at a position corresponding to the right door mirror of the own vehicle 111 . The electronic mirror cameras 109 and 110 output each data of each electronic mirror image to the electronic control unit 115 in response to a request from the electronic control unit 115 .

The navigation system 112 is a device for executing navigation to a destination. The navigation system 112 includes a GPS (Global Positioning System), a control unit, and the like. The navigation system 112 outputs vehicle information including at least the current position of the vehicle 111, the current orientation of the vehicle 111, and the current date and time to the electronic control unit 115 in response to a request from the electronic control unit 115.

The communication module 113 is a device mounted on the vehicle 111 and capable of communicating with communication targets around the vehicle 111 . The communication module 113 is, for example, a DCM (Data Communication Module). The communication module 113 transmits and receives radio waves between the own vehicle 111 and surrounding communication targets by wireless communication conforming to communication standards such as LTE (Long Term Evolution) and 5G (5th Generation). The communication target is, for example, another vehicle, a roadside device, a mobile terminal, a communication device provided in a store, or the like. The mobile terminal is, for example, a smart phone or the like carried by the driver or other passengers.

The own vehicle 111 becomes a connected car connected to the Internet by the communication module 113 . The communication module 113 acquires the latest high-definition map data and the like of the road on which the vehicle 111 travels from a server provided in the cloud. The communication module 113 outputs vehicle information including at least the current position of the vehicle 111, the current orientation of the vehicle 111, and the current date and time to the electronic control unit 115 in response to a request from the electronic control unit 115.

Temperature sensor 114 measures the temperature of windshield 116 of host vehicle 111 . The temperature sensor 114 is installed, for example, on the back side of the room mirror. The temperature sensor 114 outputs temperature data of the windshield 116 to the electronic control unit 115 in response to a request from the electronic control unit 115 . Note that the temperature sensor 114 may be incorporated in, for example, a raindrop detection device.

The electronic control unit 115 is a device having an illuminance estimation function, a light sensor function, and a solar radiation sensor function. The electronic control unit 115 is an ECU (Electronic Control Unit). The electronic control unit 115 acquires each image from each camera 101 to 110 and also acquires vehicle information from the navigation system 112 or communication module 113 .

The illuminance estimation function is a function in which the electronic control unit 115 estimates the current position of the sun and the illuminance around the vehicle 111 based on each image and vehicle information. The electronic control unit 115 has data of correlations between image pixel values and illuminance. The pixel value represents the color density and brightness of each pixel. The electronic control unit 115 acquires pixel values through image processing, and adjusts the pixel values using vehicle information about the own vehicle 111 .

That is, the electronic control unit 115 stores information such as the current date and time, the position and orientation of the vehicle 111, and how the vehicle 111 is illuminated by the sun according to the current position of the sun. reflected in the value. Then, the electronic control unit 115 acquires the illuminance corresponding to the acquired image value, thereby estimating the illuminance. Furthermore, the electronic control unit 115 uses information such as the temperature of the windshield 116 to correct the estimated illuminance.

The light sensor function is a function in which the electronic control unit 115 determines control of turning on/off the lights of the own vehicle 111 based on a plurality of images, vehicle information, and estimated illuminance. That is, the electronic control unit 115 determines whether the light is on or off. The electronic control unit 115 generates a light signal containing the content of light control. Electronic control unit 115 outputs a light signal to body ECU 117 . The body ECU 117 controls turning on/off of the lights of the own vehicle 111 according to the control contents of the light signal input from the electronic control unit 115 .

The solar radiation sensor function is a function in which the electronic control unit 115 determines control of the air conditioner of the own vehicle 111 based on a plurality of images, vehicle information, and estimated illuminance. The electronic control unit 115 estimates the amount of solar radiation around the vehicle 111 and the current position of the sun with respect to the vehicle 111 based on the brightness of the subject represented in the acquired image. In addition, the electronic control unit 115 determines control of vehicle interior air conditioning, such as blowing air volume and interior temperature, based on the amount of solar radiation and the current position of the sun.

The electronic control unit 115 generates a solar radiation signal that includes control details for the air conditioner. Alternatively, the electronic control unit 115 generates a solar radiation signal that does not include air conditioner control content. Electronic control unit 115 outputs a solar radiation signal to air conditioner ECU 118 .

The air conditioner ECU 118 controls the air conditioner of the own vehicle 111 according to the control contents of the solar radiation signal input from the electronic control unit 115 . Alternatively, the air conditioner ECU 118 controls the air conditioner of the own vehicle 111 using the solar radiation signal.

Also, the electronic control unit 115 is part of an advanced driving assistance system that performs control to assist the driving of the driver. Therefore, the electronic control unit 115 detects the situation around the vehicle 111 by performing image processing on the front image of the front camera 104 . The electronic control unit 115 receives information from sensors such as a vehicle speed sensor, a steering sensor, and an accelerator sensor. The electronic control unit 115 may acquire the information of each sensor from the body ECU 117 or directly from each sensor. The electronic control unit 115 uses vehicle information acquired from the navigation system 112 and the communication module 113 for controlling the own vehicle 111 .

The electronic control unit 115 grasps the driving situation of the own vehicle 111 based on the result of image processing of the forward image and the information of each sensor, and controls to prevent or reduce the collision of the own vehicle 111 with surrounding objects. to run. For example, when the electronic control unit 115 determines that it is necessary to brake or steer the vehicle in order to avoid or reduce contact with an object in front of the vehicle, a sudden steering warning signal or a sudden braking signal is issued. A warning signal is output to the body ECU 117 . The body ECU 117 informs the driver of the surrounding conditions of the vehicle 111 and controls the operation of the vehicle 111 based on each signal of the electronic control unit 115 . The above is the overall configuration of the illuminance estimation device 100 .

Next, the operation of the illuminance estimation device 100 will be described. Illuminance estimation apparatus 100 estimates illuminance according to a preset program. As an example, the electronic control unit 115 acquires each image from each of the cameras 101 to 110, and uses the front image of each image to estimate the illuminance in fine weather.

As shown in FIG. 3 , electronic control unit 115 acquires a forward image from front camera 104 and vehicle information from navigation system 112 . Electronic control unit 115 may obtain vehicle information from communication module 113 .

The electronic control unit 115 then obtains the current sun position based on the forward image and vehicle information. The current position of the sun is the azimuth and elevation of the sun relative to the vehicle 111 . You can estimate the height of the sun from the elevation angle.

For example, when the sun appears in the front image, the electronic control unit 115 can easily estimate the current position of the sun relative to the vehicle 111 from the position of the sun in the front image. On the other hand, when the sun is not shown in the forward image, the electronic control unit 115 estimates the directional angle from the peak of the luminance value in the horizontal direction of the sky detection range corresponding to the sky area in the forward image. In addition, the electronic control unit 115 estimates the elevation angle from information on the latitude, longitude, date and time, and the orientation of the own vehicle 111 included in the vehicle information.

The electronic control unit 115 may estimate the current position of the sun based on the latitude, longitude, date and time, and orientation of the vehicle 111 included in the vehicle information. Further, when estimating the current position of the sun, the electronic control unit 115 may estimate the amount of solar radiation for air conditioner control based on the pixel values of the forward image. Furthermore, based on the brightness of the subject represented by a plurality of peripheral images and electronic mirror images, the amount of solar radiation around the vehicle 111 and the current position of the sun relative to the vehicle 111 can be estimated. good.

After that, the electronic control unit 115 reflects each piece of information on the pixel values of the front image, and uses the data of the correlation between the pixel values and the illuminance to estimate the illuminance in fine weather. Note that the illuminance in fine weather may be estimated using the rain image, each peripheral image, and the electronic mirror image.

Further, the electronic control unit 115 corrects the estimated illuminance. Correcting the illuminance is not essential, but correcting the illuminance can improve the accuracy of the estimated illuminance. For example, the brightness value of the subject represented in the forward image can be used as the illumination correction parameter.

Specifically, the electronic control unit 115 determines the road surface or sky area in the subject represented in the forward image. Region determination is performed, for example, by a technique using semantic segmentation. Semantic segmentation is a deep learning algorithm that associates a label or category with every pixel in the forward image. The electronic control unit 115 has a DNN (Deep Neural Network) that has already learned objects such as road surfaces, sky areas, pedestrians, buildings, tunnels, and bridge girders. Therefore, the electronic control unit 115 uses the learned DNN as a dictionary, and the electronic control unit 115 selects road surfaces, sky areas, pedestrians, buildings, tunnels, bridge girders, etc., from among subjects represented in the forward image. Perform area judgment.

The electronic control unit 115 acquires the sum of the luminance values when the subject includes a road surface, an empty area, a pedestrian, a building, a tunnel, a bridge girder, or the like. Alternatively, electronic control unit 115 obtains an average value of these brightness values. The luminance value corresponds to the brightness of the subject. For example, the electronic control unit 115 acquires the sum or average of the luminance values of the road surface. Alternatively, the electronic control unit 115 may obtain the sum or average of the brightness values of two or more objects in the object. The electronic control unit 115 corrects the estimated illuminance based on the acquired sum or average of the luminance values.

Note that the electronic control unit 115 may decide to turn on/off the lights of the vehicle 111 based on a comparison between the total value or average value of the luminance values and a threshold value.

Another illumination correction parameter can be the current sun direction. The current direction of the sun is different from the current position of the sun, and corresponds to the direction of sunlight from which direction the vehicle 111 is illuminated by the sun.

For example, as shown in FIG. 4, the light of the sun 200 may illuminate the interior of the vehicle from the front right of the vehicle 111 obliquely. In this case, the part where the light of the sun 200 illuminates the driver, other passengers, and the subject of the seat becomes bright, and the part where the light of the sun 200 does not illuminate becomes dark. By determining such a brightness distribution, the current direction of the sun 200 with respect to the vehicle 111 can be estimated.

In order to estimate the current direction of the sun 200 , the electronic control unit 115 first acquires the driver image from the driver detection camera 101 and the passenger image from the passenger detection camera 102 . Subsequently, the electronic control unit 115 detects the faces of the driver and other passengers from the driver image and the passenger image.

After this, the electronic control unit 115 acquires the brightness distribution of the face. This makes it possible to see which parts of the face are bright and which parts are dark. Then, the electronic control unit 115 estimates the current direction of the sun 200 based on the acquired luminance distribution of the face. That is, the direction of sunlight of the sun 200 with respect to the own vehicle 111 is acquired. Further, the electronic control unit 115 corrects the estimated illuminance based on the obtained current direction of the sun 200 .

It should be noted that the electronic control unit 115 may estimate the current direction of the sun 200 by acquiring the luminance distribution of places other than the occupants including the driver. Also, the electronic control unit 115 may estimate the current direction of the sun 200 using either the driver image or the passenger image instead of both.

Information on the temperature of the windshield 116 of the vehicle 111 can be used as another illuminance correction parameter. The electronic control unit 115 can acquire the brightness of the subject in the image, but it is difficult to acquire the amount of heat. Therefore, the electronic control unit 115 acquires temperature information from the temperature sensor 114 . As a result, it is possible to obtain heat information about how much light is emitted from the sun 200 . Therefore, the electronic control unit 115 corrects the estimated illuminance according to the temperature value of the windshield 116 .

Information on the color of the preceding vehicle may be used as information on the amount of heat. From the relationship between the color of the preceding vehicle and the brightness of the preceding vehicle, information about how much the sun 200 is shining can be obtained. The electronic control unit 115 acquires any one of a front image, a rain image, and a peripheral image in which the front of the vehicle 111 is captured, and determines the preceding vehicle from among the subjects represented in the image. Then, the electronic control unit 115 corrects the estimated illuminance based on the color of the preceding vehicle. Alternatively, the electronic control unit 115 may correct the estimated illuminance based on both windshield 116 temperature information and the color of the preceding vehicle. If the preceding vehicle is not shown in the image, the electronic control unit 115 does not correct the illuminance based on the color of the preceding vehicle.

Furthermore, weather information such as the current weather and season can be used as another illuminance correction parameter. If the vehicle information includes current weather information such as rain or cloudiness, the electronic control unit 115 corrects the estimated illuminance based on the season and weather information corresponding to the date and time.

Furthermore, information on the tilt of the vehicle 111 and road surface unevenness can be used as another illuminance correction parameter. This is because the way the sun 200 illuminates the surroundings of the vehicle 111 changes depending on the inclination of the vehicle 111 and the unevenness of the road surface. The inclination of the own vehicle 111 and the unevenness of the road surface are detected by an inclination sensor, a height sensor, an acceleration sensor, or the like. The electronic control unit 115 acquires information about the inclination of the own vehicle 111 and road unevenness from each sensor, and corrects the estimated illuminance based on this information.

Then, the electronic control unit 115 determines control of turning on/off the lights of the vehicle 111 based on the comparison between the estimated illuminance or the corrected illuminance and the threshold. The electronic control unit 115 outputs a light signal including details of light control to the body ECU 117 . Thereby, the body ECU 117 controls turning on/off of the lights of the own vehicle 111 . Further, the electronic control unit 115 determines air conditioner control such as blowing air volume and room temperature. The electronic control unit 115 outputs a solar radiation signal including control contents of the air conditioner to the air conditioner ECU 118 . Accordingly, the air conditioner ECU 118 controls the air conditioner of the vehicle 111 .

As described above, in this embodiment, the illuminance around the host vehicle 111 is estimated using vehicle information and an image including a plurality of pieces of information. Therefore, the information for estimating the illuminance can be prevented from biasing the situation of the vehicle 111 in one direction. Therefore, the difference between the actual illuminance around the vehicle 111 and the estimated illuminance becomes small, so that the accuracy of illuminance estimation can be improved. In addition, since the estimated illuminance is corrected, the accuracy of illuminance estimation can be further improved.

Regarding the correspondence relationship between the description of this embodiment and the description of the scope of claims, the navigation system 112 and the communication module 113 correspond to the "communication unit" of the scope of claims. Also, the electronic control unit 115 corresponds to the "estimating section" in the claims.

(Second embodiment)
In this embodiment, mainly different parts from the first embodiment will be explained. In this embodiment, the electronic control unit 115 estimates the illuminance around the vehicle 111 based on the color and brightness of the vehicle body of the vehicle 111 . Alternatively, the electronic control unit 115 corrects the estimated illuminance based on the color and brightness of the body of the vehicle 111 . When correcting the illuminance, the electronic control unit 115 corrects the illuminance estimated based on information other than the color of the vehicle body and the brightness of the vehicle body.

The electronic control unit 115 has data on the correlation between the color of the vehicle body and the pixel value of the vehicle body and the illuminance. The electronic control unit 115 uses an image in which at least a part of the vehicle body of the own vehicle 111 is captured. For example, any one of the front image, the rain image, each peripheral image, and the electronic mirror image is used. As a result, the same effects as in the first embodiment can be obtained.

(Third embodiment)
In this embodiment, mainly different parts from the first and second embodiments will be explained. In this embodiment, each of the cameras 101 to 110 captures a plurality of images with different exposure times. Further, the electronic control unit 115 acquires a plurality of images with different exposure times from each of the cameras 101 to 110, and estimates the illuminance based on the plurality of images with different exposure times and the vehicle information.

Thereby, the dynamic range of the image acquired by the electronic control unit 115 can be ensured. Therefore, it is possible to suppress blown-out highlights and blocked-up shadows in an image. Also, for example, the illuminance in the range of 0 [lx] to 50,000 [lx] can be estimated. As a result, the same effects as in the first embodiment can be obtained.

Note that not all of the cameras 101 to 110 need support the dynamic range. A camera that captures images used for illuminance estimation should be adapted to the dynamic range. For example, if the front image is used for illuminance estimation, the front camera 104 is dynamic range compliant.

(Fourth embodiment)
In this embodiment, mainly different parts from the first to third embodiments will be explained. In this embodiment, the electronic control unit 115 determines the road surface or sky area from among the objects represented in the acquired image. For this reason, the electronic control unit 115 uses an image in which at least the road surface or the sky area is captured. For example, any one of a front image, a rain image, and a peripheral image in which the front of the vehicle 111 is captured is used.

As in the first embodiment, the electronic control unit 115 recognizes road surfaces, sky areas, pedestrians, buildings, tunnels, bridge girders, etc. included in the forward image by a method using semantic segmentation.

The electronic control unit 115 has data on the correlation between the total pixel value of the road surface or the sky area and the illuminance, or the correlation data between the average pixel value of the road surface or the sky area and the illuminance. The electronic control unit 115 then estimates the illuminance based on the sum of the pixel values of the road surface or sky area. Alternatively, the electronic control unit 115 estimates the illuminance based on the average value of the pixel values of the road surface or sky area. As a result, the same effects as in the first embodiment can be obtained.

(Fifth embodiment)
In this embodiment, mainly different parts from the first to fourth embodiments will be explained. In this embodiment, the electronic control unit 115 cuts out a part of the image used for estimating the illuminance, and estimates the illuminance based on the cut out image.

Since the shooting angle of view is known in advance, it is possible to grasp in advance in which region of the image the sky is located. Therefore, the electronic control unit 115 cuts out the part including the sky area from the forward image and the rain image. In other words, electronic control unit 115 narrows the angle of view of the image. Since the cropped image has a narrower range than the original image, the amount of computation for image processing can be reduced.

Note that the camera that captures the image used for illuminance estimation may cut out a part of the image and output the cut out image to the electronic control unit 115 . This makes it possible to reduce not only the image processing load but also the amount of communication and the communication load.

(Sixth embodiment)
In this embodiment, mainly different parts from the first embodiment will be explained. Since the illuminance does not change in a short period of time, it is not necessary to make determinations for each frame. Therefore, in the present embodiment, each of the cameras 101 to 110 outputs data of captured images to the electronic control unit 115 at regular intervals. For example, each camera 101 to 110 outputs image data at intervals of 0.1 to 2 seconds.

Note that all of the cameras 101 to 110 do not have to output image data at regular intervals. For example, when the front image is used for illuminance estimation, the data of the front image of the front camera 104 may be output at regular time intervals.

Then, the electronic control unit 115 estimates the illuminance based on the images acquired at regular time intervals. As a result, the amount of communication for each camera 101-110 can be reduced. In addition, it is possible to reduce the amount of communication, the communication load, and the amount of calculation for image processing of the electronic control unit 115 .

(Other embodiments)
The configuration of the illuminance estimation device 100 shown in each of the above embodiments is an example, and the configuration is not limited to the configuration shown above, and other configurations that can realize the present invention can also be adopted. For example, the electronic mirror cameras 109 and 110 may include rearview mirror cameras.

Further, although the illuminance estimation device 100 has been described as a part of the advanced driving support system, the cameras 101 to 110 and the electronic control unit 115 may be dedicated to the illuminance estimation device 100. FIG. Further, it is not necessary to have all of the cameras 101 to 110, and it is sufficient to have cameras necessary for illuminance estimation.

100 illumination estimation device 101 to 110 camera 111 vehicle 112 navigation system (communication unit)
113 communication module (communication part)
114 temperature sensor 115 electronic control unit (estimation unit)
116 wind shield 200 sun

Claims (12)

Cameras (101 to 110) mounted on a vehicle (111) for photographing an area directly or indirectly illuminated by the sun (200);
a communication unit (112, 113) for acquiring vehicle information including at least the current position of the own vehicle, the current orientation of the own vehicle, and the current date and time;
An estimation unit (115) that acquires an image from the camera, acquires the vehicle information from the communication unit, and estimates the current position of the sun and the illuminance around the vehicle based on the image and the vehicle information. )When,
An illuminance estimator, comprising: The cameras include a driver detection camera (101) that photographs a driver and an occupant detection camera (102) that photographs a passenger,
The estimation unit obtains a driver image captured by the driver detection camera and an occupant image captured by the occupant detection camera, and determines the brightness of the subject represented in the driver image or the subject represented in the occupant image. 2. The illuminance estimating device according to claim 1, which estimates the current direction of the sun with respect to said own vehicle, based on the brightness of . The cameras include a rain camera (103) for a rain sensor that captures an image of the front of the vehicle relative to the direction of travel of the vehicle, and an image of the front of the vehicle relative to the direction of travel of the vehicle. A front camera (104), a plurality of peripheral cameras (105 to 108) for photographing the front, rear, left and right sides of the vehicle with respect to the traveling direction of the vehicle, and the rear and rear sides of the vehicle. At least one of the electronic mirror cameras (109, 110) that shoots the direction,
The estimation unit includes a rain image captured by the rain camera, a forward image captured by the front camera, a plurality of peripheral images captured by the plurality of peripheral cameras, and an electronic image captured by the electronic mirror camera. At least one of the mirror images is acquired, and the amount of solar radiation around the vehicle is estimated based on the brightness of the subject represented in the acquired image, or the vehicle is used as a reference. 3. The illuminance estimating device according to claim 1, which estimates the current position of the sun. The communication unit includes a navigation system (112) mounted on the vehicle, or a communication module (113) mounted on the vehicle and capable of communicating with communication targets around the vehicle. ,
The estimation unit estimates the current position of the sun based on the vehicle information obtained from the navigation system or the communication module, and corresponds to the date and time when the vehicle information includes current weather information. 4. The illuminance estimation device according to claim 1, wherein said estimated illuminance is corrected based on said season and said weather information. The estimation unit estimates the illuminance around the vehicle based on the color of the vehicle body and the brightness of the vehicle body represented in an image in which the vehicle body is photographed, or the estimated illuminance. 5. The illuminance estimation device according to any one of claims 1 to 4, which corrects the . The camera captures a plurality of images with different exposure times as the images,
The estimation unit acquires the plurality of images with the exposure times changed from the camera, and estimates the illuminance based on the plurality of images with the exposure times changed and the vehicle information. 6. The illuminance estimation device according to any one of 1 to 5. The estimating unit determines any one of a road surface, a sky area, a building, a tunnel, and a bridge girder from objects represented in the image, and determines whether the road surface, the sky area, 7. The illuminance according to any one of claims 1 to 6, wherein when any one of the building, the tunnel, and the bridge girders is included, the estimated illuminance is corrected based on the brightness of those included. estimation device. a temperature sensor (114) for measuring the temperature of the windshield (116) of the vehicle;
The estimating unit acquires temperature information from the temperature sensor, determines a preceding vehicle from among subjects represented in the image, and corrects the estimated illuminance based on the temperature information, or 8. The method according to any one of claims 1 to 7, wherein the estimated illuminance is corrected based on the color of the preceding vehicle, or the estimated illuminance is corrected based on the temperature information and the color of the preceding vehicle. illuminance estimation device. wherein the estimation unit determines control of turning on/off the lights of the own vehicle or determines control of an air conditioner of the own vehicle based on the image, the vehicle information, and the estimated illuminance. Item 9. The illuminance estimation device according to any one of Items 1 to 8. The estimating unit determines a road surface or sky area from among subjects represented in the image, and based on a total value of pixel values of the road surface or the sky area, or the road surface or the sky area. 10. The illuminance estimation device according to claim 1, wherein said illuminance is estimated based on an average value of pixel values of . The illuminance estimation device according to any one of claims 1 to 10, wherein the estimation unit cuts out a part of the image and estimates the illuminance based on the cut out image. The camera outputs data of the captured image to the estimation unit at regular time intervals,
The illuminance estimation device according to any one of claims 1 to 11, wherein the estimation unit estimates the illuminance based on images acquired from the camera at regular time intervals.

Images