JP2022116377A - Weather determination device and weather determination method - Google Patents

Abstract

To provide a weather determination device and a weather determination method which can accurately determine the weather even when a three-dimensional object is photographed in an image used for determination of the weather.SOLUTION: A weather determination device comprises: an actual edge intensity calculation part which calculates an actual edge intensity being an edge intensity in a region where a three-dimensional object in an image is detected; a three-dimensional object type determination part which determines a type of the three-dimensional object from a feature amount in the three-dimensional object; a type edge intensity acquisition part which acquires a type edge intensity on the basis of a type edge intensity table that previously determines the edge intensity in the good visual field weather in which the visual field is good that is corresponding to a distance and the type of the three-dimensional object determined by the three-dimensional object type determination part; and a weather determination part which determines that the weather within a range of the image is the poor visual field weather in which the visual field is poor when a difference between the actual edge intensity calculated by the actual edge intensity calculation part and the type edge intensity acquired by the type edge intensity acquisition part is larger than a predetermined threshold, and outputs a determination result.SELECTED DRAWING: Figure 1

Description

The present invention relates to a weather determination device and a weather determination method.

In recent years, automation of vehicle control has attracted attention, and for example, automation of light distribution control is also being studied. In order to appropriately perform such light distribution control, it is necessary to accurately detect the weather around the vehicle, such as fog, snow, and heavy rain, and perform control according to the detected weather.

As a technology related to such weather detection, for example, Patent Document 1 discloses an in-vehicle fog judgment device provided with fog judgment means for judging fog based on an image taken by an in-vehicle camera mounted on the vehicle. a road area determining means for determining a road area on which the vehicle is traveling in an image captured by the vehicle-mounted camera; distant road surface area determining means for determining a distant road surface area which is a road surface area located a predetermined distance away from the vehicle, wherein the fog determination means is based on the luminance of the distant road surface area determined by the distant road surface area determining means. A fog determination is disclosed.

JP 2007-057331 A

In the conventional technology described above, an image captured by an in-vehicle camera is divided into a near road surface area, a distant road surface area, and a sky area, and fog is generated by comparing the near road surface area with the distant road surface area and the distant road surface area with the sky area. is detected. However, when using an image captured by an in-vehicle camera, weather such as fog and cloudy weather on the windshield in front of the camera are detected in the same manner, and it is conceivable that weather detection cannot be performed appropriately.

In addition, if three-dimensional objects (cars, roadside trees, road signs, etc.) are reflected in each divided area, noise will be added to the comparison between the near road surface area and the distant road surface area, and the distant road surface area and the sky area. Accurate detection is not possible.

SUMMARY OF THE INVENTION The present invention has been made in view of the above, and provides a weather determination device and weather determination method capable of accurately determining weather even when a three-dimensional object is captured in an image used for weather determination. The purpose is to

The present application includes a plurality of means for solving the above problems. One example is a three-dimensional object detection unit that detects a three-dimensional object and the distance to the three-dimensional object from an image captured by an imaging unit of a vehicle. a three-dimensional object type determination unit that determines the type of the three-dimensional object from the feature amount of the three-dimensional object; and an actual edge strength calculation unit that calculates the actual edge strength, which is the edge strength in the area of the image where the three-dimensional object is detected. and, based on a type edge strength table that predetermines the edge strength during weather with good visibility corresponding to the type and distance of the three-dimensional object determined by the three-dimensional object type determination unit, the type edge strength is determined. When the difference between the type edge strength obtained by the type edge strength obtaining unit and the type edge strength obtained by the type edge strength obtaining unit and the actual edge strength calculated by the actual edge strength calculation unit is larger than a predetermined threshold, A weather determination unit that determines that the weather in the range of the image is low visibility weather, and outputs a determination result.

According to the present invention, weather can be accurately determined even when a three-dimensional object is captured in an image used for weather determination.

It is a functional block diagram which shows the processing content of a weather determination apparatus. FIG. 4 is a diagram showing the relationship between the distance from the host vehicle to a three-dimensional object and the edge strength; It is a figure which shows that the relationship between the distance from the own vehicle to a three-dimensional object, and edge strength changes with weather. 5 is a flowchart showing the processing contents of an actual edge strength calculator; 9 is a flow chart showing the processing contents of a type edge strength acquisition unit; It is a flow chart which shows the processing content of the poor visibility weather judgment part.

In the present embodiment, a three-dimensional object in an image can Noting that there is a difference in the feature amount (e.g., edge strength) of the three-dimensional object in the image, the weather is determined based on the feature amount of the three-dimensional object in the image. The weather can be determined with high accuracy even when

An embodiment of the present invention will be described below with reference to FIGS. 1 to 6. FIG.

FIG. 1 is a functional block diagram showing the processing contents of the weather determination device according to this embodiment.

In FIG. 1, the weather determination device 100 includes a three-dimensional object detection unit 110 that detects a three-dimensional object and the distance to the three-dimensional object from an image captured by an imaging unit of a vehicle, and an actual edge strength of the three-dimensional object in the image. When the difference between the actual edge strength and the type edge strength is larger than a predetermined threshold, the weather in the range of the image is poor visibility weather. and a poor visibility weather determination unit 130 that determines and outputs the determination result.

Further, the edge strength calculation unit 120 includes an actual edge strength calculation unit 121 that calculates the actual edge strength, which is the edge strength in the region where the three-dimensional object is detected in the image, and a three-dimensional object that determines the type of the three-dimensional object from the feature amount of the three-dimensional object. Based on an object type determination unit 122 and a type edge strength table that predetermines edge strengths in good visibility and good weather conditions corresponding to the types and distances of three-dimensional objects determined by the three-dimensional object type determination unit 122, It has a type edge strength obtaining unit 123 that obtains type edge strengths and a type edge strength table storage unit 124 that stores a type edge strength table.

For example, the three-dimensional object detection unit 110 detects a three-dimensional object in the image based on an image captured by an imaging device such as a stereo camera installed so as to image the front of the vehicle, and detects the detected three-dimensional object. A distance from the imaging device (the relative position of the imaging device in the vehicle can be obtained from design information or the like, so it can also be called the distance of the three-dimensional object from the vehicle) is calculated.

The actual edge strength calculation unit 121 calculates the edge strength in the region where the three-dimensional object detected in the image exists. Edge strength is one of the feature quantities of a three-dimensional object in an image.

The three-dimensional object type determination unit 122 determines the type of the three-dimensional object detected in the image, and outputs the determination result together with the distance information to the three-dimensional object obtained by the three-dimensional object detection unit 110 . The types of three-dimensional objects to be determined are, for example, vehicles, roadside trees, and road signs. Although the method for determining the type of the three-dimensional object is not particularly limited, for example, in addition to a method such as pattern matching, various feature amounts (for example, the edge strength calculated by the actual edge strength calculation unit 121 also corresponds). A method of determination or the like may be used.

The type edge strength acquisition unit 123 reads and acquires the corresponding type edge strength table from the type edge strength table storage unit 124 based on the type of the three-dimensional object determined by the three-dimensional object type determination unit 122, and obtains the corresponding type edge strength table. Based on the distance information, the edge strength of the three-dimensional object (that is, the type edge strength) estimated in the weather defined by the type edge strength table (here, good visibility weather) is obtained and output.

For the type edge strength table stored in the type edge strength table storage unit 124, the relationship between the distance to the three-dimensional object and the edge strength in good visibility weather (for example, when the weather is fine) is experimentally obtained in advance for each type of three-dimensional object. It is a thing. The type edge strength table storage unit 124 stores a type edge strength table for each three-dimensional object of at least a part of the three-dimensional object whose type is to be determined by the three-dimensional object type determination unit 122 .

In the present embodiment, the case of using the type edge strength table for weather with good visibility has been described as an example, but the present invention is not limited to this. It may be configured to store them and read them according to the type of three-dimensional object.

When the difference between the actual edge strength calculated by the actual edge strength calculation section 121 and the type edge strength obtained by the type edge strength acquisition section 123 is greater than a predetermined threshold, the poor visibility weather determination section 130 determines the range of the image. is determined to be poor visibility weather with poor visibility, and the determination result is output.

Here, the basic principle of weather determination in this embodiment will be described.

FIG. 2 is a diagram showing the relationship between the distance from the host vehicle to the three-dimensional object and the edge strength. Further, FIG. 3 is a diagram showing that the relationship between the distance from the host vehicle to the three-dimensional object and the edge strength varies depending on the weather.

As shown in FIG. 2, the edge strength changes according to the distance to the three-dimensional object, and the edge strength decreases as the distance increases. In addition, as shown in FIG. 3, the manner in which the edge strength changes with respect to the distance to the three-dimensional object also differs depending on the road surface conditions and the weather. In FIGS. 2 and 3, as an example, a case where roadside trees, preceding vehicles, etc. are within the imaging range of the camera is exemplified, but in reality, one or more features such as road signs may be present. . As shown in FIG. 3, when the weather is poor visibility, the amount of edge strength attenuation is greater when the distance is longer than when the weather is good visibility. Therefore, it is possible to determine whether or not the visibility is poor by recording the edge strength for each three-dimensional object type and distance in good visibility weather and comparing it with the edge strength of the detected three-dimensional object.

FIG. 4 is a flow chart showing the processing contents of the actual edge strength calculation section, FIG. 5 is a flow chart showing the processing contents of the type edge strength acquisition section, and FIG.

In FIG. 4, the actual edge strength calculation unit 121 acquires information on the position where the three-dimensional object is detected in the image by the three-dimensional object detection unit 110, and extracts the rectangular area where the three-dimensional object is captured based on the information. (step S100), and the edge intensity is calculated and output for the obtained rectangular area (step S110).

In FIG. 5, the type edge strength acquisition unit 123 acquires the determination result of the type of the three-dimensional object determined by the three-dimensional object type determination unit 122 (step S200), and the three-dimensional object calculated by the three-dimensional object detection unit 110 (step S210), and based on the obtained determination result of the type of the three-dimensional object, the type edge strength table read from the type edge strength table storage unit 124 is used to obtain the type edge strength based on the distance to the three-dimensional object. (step S220).

In FIG. 6, the poor visibility weather determining unit 130 acquires the actual edge strength calculated by the actual edge strength calculating unit 121 (step S300), and acquires the type edge strength obtained by the type edge strength obtaining unit 123. (Step S310), it is determined whether or not the difference between the actual edge strength and the type edge strength is equal to or greater than a predetermined threshold value (Step S320). If the determination result in step S320 is YES, it is determined that the weather is poor visibility (step S321), and the process ends. If the determination result in step S320 is NO, it is determined that the weather is not poor visibility weather (in this case, the weather is good visibility) (step S322), and the process ends.

In the present embodiment configured as described above, the weather is determined based on the difference in the edge strength of the three-dimensional object in the image between when the visibility is good and when the visibility is poor. Weather can be accurately determined even when a three-dimensional object is imaged in the image used for determination.

<Appendix>
It should be noted that the present invention is not limited to the above embodiments, and includes various modifications and combinations within the scope of the invention. Moreover, the present invention is not limited to those having all the configurations described in the above embodiments, and includes those having some of the configurations omitted. Further, each of the above configurations, functions, etc. may be realized by designing a part or all of them, for example, with an integrated circuit. Moreover, each of the above configurations, functions, etc. may be realized by software by a processor interpreting and executing a program for realizing each function.

DESCRIPTION OF SYMBOLS 100... Weather determination apparatus 110... Solid object detection part 120... Edge strength calculation part 121... Actual edge strength calculation part 122... Solid object type determination part 123... Type edge strength acquisition part 124... Type edge strength table storage unit, 130 ... poor visibility weather determination unit

Claims (2)

a three-dimensional object detection unit that detects a three-dimensional object from an image captured by an imaging unit of a vehicle and calculates a distance to the three-dimensional object;
an actual edge strength calculation unit that calculates the actual edge strength, which is the edge strength in the region of the image where the three-dimensional object is detected;
a three-dimensional object type determination unit that determines the type of the three-dimensional object from the feature amount of the three-dimensional object;
A type edge strength is acquired based on a type edge strength table that predetermines edge strengths in weather with good visibility corresponding to the type and distance of the three-dimensional object determined by the three-dimensional object type determination unit. a type edge strength acquisition unit;
When the difference between the actual edge strength calculated by the actual edge strength calculation unit and the type edge strength obtained by the type edge strength obtaining unit is larger than a predetermined threshold, the weather in the range of the image is poor visibility. and a weather determination unit that determines that the weather is poor visibility weather and outputs a determination result. a procedure for detecting a three-dimensional object and a distance to the three-dimensional object from an image captured by an imaging unit of a vehicle;
a step of calculating an actual edge strength, which is the edge strength in the area of the image where the three-dimensional object is detected;
a procedure for determining the type of the three-dimensional object from the feature amount of the three-dimensional object;
a procedure for obtaining a type edge strength based on a type edge strength table that predetermines edge strengths in weather with good visibility corresponding to the type and distance of the three-dimensional object;
a step of determining that the weather in the range of the image is poor visibility weather with poor visibility when the difference between the actual edge strength and the type edge strength is greater than a predetermined threshold value, and outputting a determination result; A weather determination device comprising:

Images