JP2019219816A - Image processing system and lighting fixture for vehicle - Google Patents

Abstract

To provide a novel technique discriminating a condition in front of a vehicle with a relatively simple method.SOLUTION: An image processing system comprises a road shape estimation unit defining a region including a roadway outside line from a picked-up image obtained by imaging a front of a vehicle and estimating a road shape on the basis of a shape of the region and a reference point calculation unit calculating a far reference point on the basis of an intersection of a line extending a diagonal of the region and a horizontal line in the picked-up image.SELECTED DRAWING: Figure 5

Description

  The present invention relates to an image processing device used for an automobile or the like.

  2. Description of the Related Art In recent years, various attempts have been made to determine a surrounding environment or an object based on surrounding information acquired by a camera or a sensor mounted on a vehicle, and to perform vehicle control according to the environment or the object. The vehicle control includes various controls such as braking control, drive control, operation control, and light distribution control.

  For example, when performing light distribution control according to a situation in front of the vehicle, it is difficult to perform accurate light distribution control unless the shape of the road, which is one of the environments in which the vehicle travels, is known. Therefore, various methods have been devised for estimating the road shape itself. For example, there has been devised a partition line recognition device capable of highly accurately estimating the other white line based on one white line detected from an acquired image (see Patent Document 1).

JP 2003-44836 A

  In the light distribution control, it is relatively easy to determine the attribute of a light spot (a light emitting body such as a vehicle or a streetlight, or a reflector such as a delineator or a sign) from the acquired image. However, in an acquired image, distant light spots tend to be dark, and the light spots often approach each other. Therefore, it is difficult to determine the attribute of the distant light spot. Therefore, if a high-precision camera or an arithmetic device having a high image processing capability is used, it is relatively easy to determine the attribute of a distant light spot, but this increases the cost of the entire light distribution control system.

  The present invention has been made in view of such a situation, and an object of the present invention is to provide a new technique for determining a situation ahead of a vehicle by a relatively simple method.

  In order to solve the above problem, an image processing apparatus according to an aspect of the present invention determines a region including a roadway outside line from a captured image obtained by capturing an image in front of a vehicle, and estimates a road shape based on the shape of the region. A shape estimating unit; and a reference point calculating unit that calculates a distant reference point based on an intersection of a straight line obtained by extending a diagonal line of the area and a horizontal line in the captured image.

  According to this aspect, a distant reference point, for example, a vanishing point, can be calculated using the shape of the area in which the road shape is estimated. Therefore, the situation in front of the vehicle can be determined by a relatively simple method from the captured image of the front of the vehicle.

  The road shape estimating unit may determine a rectangular region as the region, and estimate the road shape based on information correlated with the aspect ratio of the rectangular region. Thereby, the road shape can be estimated based on the shape of the rectangular area including the roadway outside line.

  The information processing apparatus may further include a determination unit that determines a situation ahead of the vehicle using the distant reference point. The determining unit may determine that the light spot moving diagonally downward from the distant reference point is an oncoming vehicle. Thereby, the oncoming vehicle ahead of the vehicle can be easily determined.

  The information processing apparatus may further include a determination unit that determines a situation ahead of the vehicle using the distant reference point. The determination unit may determine that the light spot that stays between the distant reference point and the host vehicle is the preceding vehicle. Thus, the preceding vehicle ahead of the vehicle can be easily determined.

  An image processing device, a headlight unit that irradiates the front of the vehicle, and a light distribution control unit that controls light distribution of the headlight unit according to an attribute of a light spot determined by the image processing device may be provided. . Thereby, it is possible to realize an appropriate light distribution for an oncoming vehicle or a preceding vehicle in front of the vehicle.

  Note that any combination of the above-described components and any conversion of the expression of the present invention among methods, apparatuses, systems, and the like are also effective as embodiments of the present invention.

  According to the present invention, the situation ahead of the vehicle can be determined by a relatively simple method.

It is a schematic diagram showing the appearance of the vehicle to which the vehicular lamp according to the present embodiment is applied. It is a block diagram showing a schematic structure of a vehicular lamp according to the present embodiment. 5 is a flowchart illustrating a light distribution control method including a situation determination process in front of the vehicle according to the present embodiment. 4 is a flowchart showing details of step S10 shown in FIG. FIG. 5A is a diagram schematically showing image data when the road shape ahead of the vehicle is a straight line, and FIG. 5B is a diagram schematically showing image data when the road shape ahead of the vehicle is a left curve. FIG. 5C is a diagram schematically showing image data when the road shape ahead of the vehicle is a right curve. It is a schematic diagram which shows the situation where many light-emitting objects exist in the road of the left curve at night seen from the front monitoring camera. FIG. 7A is a diagram schematically illustrating image data when the road shape ahead of the vehicle is a straight line without a gradient, and FIG. 7B is an image when the road shape ahead of the vehicle is a straight line on an uphill. FIG. 7C is a diagram schematically illustrating data, and FIG. 7C is a diagram schematically illustrating image data when the road shape ahead of the vehicle is a downhill.

  Hereinafter, the present invention will be described based on embodiments with reference to the drawings. The same or equivalent components, members, and processes shown in each drawing are denoted by the same reference numerals, and the repeated description will be omitted as appropriate. In addition, the embodiments do not limit the invention, but are exemplifications, and all features and combinations described in the embodiments are not necessarily essential to the invention.

(Vehicle lighting)
FIG. 1 is a schematic diagram showing the appearance of a vehicle to which the vehicle lamp according to the present embodiment is applied. As shown in FIG. 1, a vehicle 10 according to the present embodiment includes a headlight unit 12 that irradiates the front of the vehicle, a control system 14 that controls light irradiation by the headlight unit 12, and a traveling of the vehicle 10. Various sensors that detect information indicating the situation and output the detection signal to the control system 14, a forward monitoring camera 16 that monitors the front of the vehicle, and an antenna that receives an orbit signal from a GPS satellite and outputs it to the control system 14. And 18.

  As various sensors, for example, a steering sensor 22 that detects a steering angle of the steering wheel 20, a vehicle speed sensor 24 that detects a vehicle speed of the vehicle 10, and an illuminance sensor 26 that detects illuminance around the own vehicle are provided. And these sensors 22, 24, 26 are connected to the control system 14 described above.

  In order to use the front monitoring camera 16 as light distribution control of a headlight unit (headlight), it is required that an object ahead of the vehicle can be identified at night. However, there are various objects that exist in front of the vehicle. Objects such as oncoming vehicles and preceding vehicles that require light distribution control in consideration of glare, and glare such as road lighting and delineators (gaze guidance signs) are not considered. Some objects need only to perform optimal light distribution control for the host vehicle.

  In order to realize such a light distribution control of the headlight unit, a light emitting body such as a front running vehicle (an oncoming vehicle or a preceding vehicle) traveling in front of the own vehicle, a road light, or a light source such as a delineator is used. It is preferable to use a camera that detects the reflector. In addition, it is more preferable to have a function of determining a situation in front of the vehicle such as an attribute of a light emitting body or a light reflecting body detected as an object or a road shape. Here, the attribute distinguishes, for example, whether the forward light-emitting body or light reflector is a preceding vehicle or a facility attached to a road. More specifically, if the luminous body or the like is a vehicle, it is a preceding vehicle or an oncoming vehicle. If the luminous body or the like is a facility attached to a road, it is a road lighting, a delineator, or another luminous facility. (For example, store lighting, advertisement, etc.) or a traffic signal. With respect to the road shape, it is determined whether the vehicle is going straight, a right curve, a left curve, an uphill, or a downhill. In addition, the curvature of the curve and the gradient of the slope can be determined in several ranges.

  The headlamp unit applicable to the present embodiment is not particularly limited as long as it can change the light distribution of light to be irradiated according to the attribute of an object existing ahead. For example, a head lamp using a halogen lamp, a gas discharge head lamp, or a semiconductor light emitting element (LED, LD, EL) can be employed. In the present embodiment, a headlamp unit having a configuration in which a part of a light distribution pattern can be made non-irradiated so as not to give glare to a leading vehicle will be described as an example. Note that a configuration in which a part of the light distribution pattern can be made non-irradiated includes a configuration in which a shade is driven to partially block light from a light source, and a configuration in which a part of a plurality of light emitting units is turned off. It is.

  The headlight unit 12 includes a pair of left and right headlight units 12R and 12L. The headlamp units 12R and 12L have the same configuration except that the internal structure is symmetrical. A low beam lamp unit 28R and a high beam lamp unit 30R are provided in a right lamp housing, and a low beam lamp unit is provided in a left lamp housing. Lamp unit 28L and a high beam lamp unit 30L are arranged.

  The control system 14 does not irradiate the headlight units 12R and 12L provided on the left and right sides of the front part of the vehicle, that is, a partial area of the light distribution pattern, based on the input outputs of the various sensors. Controls the headlight unit 12 whose light distribution characteristics can be changed.

  Next, the vehicle lamp according to the present embodiment will be described. FIG. 2 is a block diagram showing a schematic configuration of vehicle lamp 110 according to the present embodiment. The vehicle lamp 110 includes headlight units 12R and 12L, and a control system 14 that controls light irradiation by the headlight units 12R and 12L. Then, the vehicle lighting device 110 determines, in the control system 14, a situation in front of the vehicle such as an attribute of an object existing in front of the vehicle and a road shape, determines a light distribution control condition based on the situation, and determines the determined light distribution. Light irradiation by the headlight units 12R and 12L is controlled based on the control conditions.

  Therefore, a forward monitoring camera 16 for acquiring a captured image in front of the vehicle including a driver's view target is connected to the control system 14 according to the present embodiment. In addition, a steering sensor 22, a vehicle speed sensor 24, and an illuminance sensor 26 for detecting steering information and vehicle speed, which are referred to when determining the traveling state of the vehicle, are connected.

(Control system)
The control system 14 includes an image processing ECU 32, a light distribution control ECU 34, and a GPS navigation ECU 36. Various ECUs and various in-vehicle sensors are connected by an in-vehicle LAN bus so that data can be transmitted and received. The image processing ECU 32 determines an attribute of an object existing ahead based on data of a captured image acquired by the front monitoring camera 16 and various in-vehicle sensors. The light distribution control ECU 34 determines light distribution control conditions suitable for the traveling environment in which the vehicle is placed based on information from the image processing ECU 32 and various in-vehicle sensors, and sends the control signal to the headlight units 12R and 12L. Output.

  Light distribution of the headlight units 12R and 12L is controlled by inputting a control signal output from the light distribution control ECU 34 to a drive device for optical components and a lighting control circuit for a light source. The forward monitoring camera 16 is a single-lens zoom camera having an image sensor such as a CCD or a CMOS, and uses the image data to provide information on road alignment information necessary for driving, road-related facilities, and the presence and location of oncoming vehicles and preceding vehicles. Get etc.

(Situation determination process in front of the vehicle)
It is difficult to instantaneously determine the light spots of the vehicle (head lamps and tail lamp lights) and other light spots (street lights, reflectors, advertisement displays, store lighting, etc.) from nighttime images. In particular, the light spot of a distant vehicle has low brightness and is more difficult to distinguish. In addition, in order to reduce undetected light spots in a distant place, it is one idea to detect a predetermined distant area in which the distant detection is performed, and to separately perform a light spot determination process suitable for the distant area. For that purpose, it is necessary to accurately detect a distant region. However, the position of the distant area changes depending on the road shape (whether it is a curve or a slope).

  Therefore, the present inventors have devised a new image processing apparatus that estimates a road shape and calculates a reference point in a distant region from the road shape. This image processing apparatus calculates a distant reference point and a distant region including the distant reference point by the following method, and can perform image processing with a relatively small amount of calculation. The image processing device according to the present embodiment can determine the situation in front of the vehicle such as the type of light spot and the road shape using such simple image processing.

  First, an outline of a situation determination process in front of the vehicle in the image processing ECU according to the present embodiment will be described. The situation determination process in front of the vehicle according to the present embodiment calculates the distant reference point based on the shape of the area including the roadway outer line included in the captured image obtained by capturing the front of the vehicle, and uses the distant reference point. This is to determine the situation in front of the vehicle. Here, the roadway outside line is a so-called white line indicating a boundary between the traveling lane and the sidewalk, and refers to a white line on the left side of the traveling lane when the vehicle travels on the left side.

  FIG. 3 is a flowchart illustrating a light distribution control method including a situation determination process in front of the vehicle according to the present embodiment. FIG. 4 is a flowchart showing details of step S10 shown in FIG.

(Estimation of road shape)
The determination of the situation in front of the vehicle is mainly executed by the image processing ECU 32 shown in FIG. 2, and the light distribution control is mainly executed by the light distribution control ECU 34. The image processing ECU 32 according to the present embodiment estimates the road shape corresponding to the light spot based on the characteristic information such as the shape and position of the road outside line included in the captured image information (S10 in FIG. 3). .

  The estimation of the road shape shown in step S10 is performed, for example, in the process shown in FIG. First, the image information acquisition unit 38 acquires data of a captured image of the front of the vehicle captured by the front monitoring camera 16 (S100), and binarizes the image data (S102). FIG. 5A is a diagram schematically showing image data when the road shape ahead of the vehicle is a straight line, and FIG. 5B is a diagram schematically showing image data when the road shape ahead of the vehicle is a left curve. FIG. 5C is a diagram schematically showing image data when the road shape ahead of the vehicle is a right curve. In the following description, the case where the road shape is immediately before will be described.

  Next, the white line detection unit 40 determines whether or not there is a white line drawn on the left side of the traveling lane of the host vehicle (hereinafter, appropriately referred to as “left white line”) from the binarized image data. It is detected (S104). At this time, in order to prevent erroneous detection of the left white line, information such as the size, position, and width of the left white line is used as a detection condition. If the left white line has not been detected (No in S104), the process ends once, and the captured image data is acquired again.

  On the other hand, for example, when the left white line L1 is detected in the image data shown in FIG. 5A (Yes in S104), the road shape estimation unit 42 determines an area including a part of the left white line (S106). Then, the road shape is estimated based on the shape of the area (S108). Specifically, a rectangular area R1 including a part of the left white line L1 is determined, and a road shape is estimated based on an aspect ratio A (shape of the rectangular area R1) calculated by vertical Y / horizontal X. For example, when the aspect ratio A is 0.8 or more and 1.2 or less, the road shape estimation unit 42 estimates that the road shape is a straight line.

  In addition, the road shape estimating unit 42 determines a rectangular region R2 including a part of the left white line L2 in the image data illustrated in FIG. 5B, and determines a road based on the aspect ratio A calculated by vertical Y / horizontal X. Estimate the shape. For example, when the aspect ratio A is larger than 1.2, the road shape estimating unit 42 estimates that the road shape is a left curve.

  In addition, the road shape estimating unit 42 determines a rectangular region R3 including a part of the left white line L3 in the image data illustrated in FIG. 5C, and determines the road based on the aspect ratio A calculated by vertical Y / horizontal X. Estimate the shape. For example, when the aspect ratio A is smaller than 0.8, the road shape estimating unit 42 estimates that the road shape is a right curve.

  After the road shape is estimated according to the flowchart shown in FIG. 4, step S12 shown in FIG. 3 is executed. When the road shape is a straight line, the reference point calculation unit 44 determines whether the straight line L1 ′ (which substantially matches the left white line L1), which is an extension of the diagonal line of the rectangular area R1 shown in FIG. 5A, and the horizontal line H1 in the captured image P1. The distant reference point (the vanishing point V1) is calculated based on the intersection C1 of (1) (S12 in FIG. 4). The horizontal line according to the present embodiment is, for example, a boundary between the ground and the sky, and is a boundary where the density and color rapidly change in image data.

  In addition, when the road shape is a left curve, the reference point calculation unit 44 determines based on an intersection C2 between a straight line L2 ′ extending the diagonal line of the rectangular area R2 shown in FIG. 5B and a horizontal line H2 in the captured image P2. To calculate the distant reference point. The straight line L2 'is a diagonal line of the two diagonal lines of the rectangular region R2 that intersects the left white line L2. Since the intersection C2 is shifted to the image center side from the vanishing point V2 in the road shape shown in FIG. 5B, the intersection C2 is corrected in consideration of, for example, the aspect ratio A that changes depending on the curvature of the curve, and the vanishing point is corrected. V2 may be calculated.

  In addition, when the road shape is a left curve, the reference point calculation unit 44 determines based on an intersection C3 between a straight line L3 ′ extending a diagonal line of the rectangular area R3 shown in FIG. 5C and a horizontal line H3 in the captured image P3. To calculate the distant reference point. Since the intersection C3 is shifted to the image center side from the vanishing point V3 in the road shape shown in FIG. 5C, the intersection C3 is corrected by taking into account, for example, the aspect ratio A that changes depending on the curvature of the curve. V3 may be calculated.

  As described above, the image processing ECU 32 according to the present embodiment can calculate a distant reference point, for example, a vanishing point, using the shape of the area in which the road shape is estimated. Therefore, the situation in front of the vehicle can be determined by a relatively simple method from the captured image of the front of the vehicle.

  When the distant reference point (or vanishing point) is calculated by the reference point calculation unit 44, the image processing ECU 32 determines the situation ahead of the vehicle using the distant reference point (S14 in FIG. 3). For example, when the distant reference point is substantially in the center area of the image, the left area of the image, and the right area of the image, the road on which the host vehicle travels is determined to be straight ahead, a left curve, or a right curve.

  Next, determination of a preceding vehicle or an oncoming vehicle traveling ahead of the own vehicle will be described. FIG. 6 is a schematic diagram illustrating a situation in which a large number of light-emitting objects are present on a left curve road at night as viewed from a front monitoring camera.

  When the vehicle travels, the light spots L10 and L11 indicating the oncoming vehicle 50 shown in FIG. 6 move obliquely downward from the distant reference point (the vanishing point V2). Therefore, the determination unit 48 determines that the light spot moving diagonally downward from the distant reference point is an oncoming vehicle. Thereby, the oncoming vehicle 50 ahead of the vehicle can be easily determined. At this time, the image processing ECU 32 may determine the oncoming vehicle in consideration of conditions such as the brightness, area, and color of the light spot (head lamp).

  When the host vehicle travels, the intersections L12 and L13 indicating the preceding vehicle 52 shown in FIG. 6 approach or move away from the distant reference point and the host vehicle. Therefore, the determining unit 48 determines that the light spot that stays between the distant reference point and the host vehicle is the preceding vehicle 52. Thus, the preceding vehicle ahead of the vehicle can be easily determined. At this time, the image processing ECU 32 may determine the preceding vehicle in consideration of conditions such as the brightness, area, and color of the light spot (tail lamp).

(Light distribution control)
The vehicular lamp 110 according to the present embodiment includes an image processing ECU 32, and a light distribution control ECU 34 that controls the light distribution of the headlight unit 12 according to the attribute of the light spot determined by the image processing ECU 32. I have. Accordingly, it is possible to achieve appropriate light distribution for an oncoming vehicle or a preceding vehicle in front of the vehicle.

  Specifically, the image processing ECU 32 determines the attribute of the object corresponding to the light spot based on characteristic information such as the position, movement, size, brightness, and locus of the plurality of light spots included in the captured image information. Is determined. The light distribution control ECU 34 sets a light spot determined by the image processing ECU 32 to be an object requiring light distribution control in consideration of glare, such as an oncoming vehicle or a preceding vehicle, as a target of light distribution control of the headlight unit 12, The headlamp unit 12 determines a light spot determined by the image processing ECU 32 as an object other than the preceding vehicle (lights attached to roads such as road lighting, delineators, store lighting, and advertisements) that do not require light distribution control in consideration of glare. (S16 in FIG. 3).

  In other words, the light distribution control ECU 34 controls the headlamp unit 12 so as to irradiate a range including a light spot determined not to be a vehicle in which the influence of glare needs to be considered, thereby enabling a visual recognition in front of the vehicle. It is possible to perform light distribution control with improved performance. As described above, the vehicular lamp 110 according to the present embodiment enables appropriate light distribution control according to the attribute of the object ahead of the vehicle without imposing a special operation burden on the driver.

(Estimation of slope)
The above-described image processing ECU 32 estimates whether or not the road shape is a curve, but can also estimate whether or not the road shape is a slope.

  FIG. 7A is a diagram schematically illustrating image data when the road shape ahead of the vehicle is a straight line without a gradient, and FIG. 7B is an image when the road shape ahead of the vehicle is a straight line on an uphill. FIG. 7C is a diagram schematically illustrating data, and FIG. 7C is a diagram schematically illustrating image data when the road shape ahead of the vehicle is a downhill.

  The image processing ECU 32 estimates whether or not the road is a slope by using the detected left white line, as in the case where the road shape is estimated to be a curve. In the image data shown in FIG. 7A, when a straight left white line L4 is detected, a rectangular region R4 including a part of the left white line L4 is determined, and the aspect ratio A (rectangular Y / horizontal X) is calculated. The road shape is estimated based on the shape of the region R4). For example, when the left white line is a straight line and the aspect ratio A is 0.8 or more and 1.0 or less, the road shape estimating unit 42 estimates that the road shape is flat with no gradient.

  Further, the road shape estimating unit 42 determines a rectangular region R5 including a part of the straight left white line L5 in the image data shown in FIG. 7B, and based on the aspect ratio A calculated by the vertical Y / horizontal X. To estimate the road shape. For example, when the aspect ratio A is larger than 1.0, the road shape estimating unit 42 estimates that the road shape is an uphill.

  Further, the road shape estimating unit 42 determines a rectangular area R6 including a part of the straight left white line L6 in the image data shown in FIG. 7C, and based on the aspect ratio A calculated by the vertical Y / horizontal X. To estimate the road shape. For example, when the aspect ratio A is smaller than 0.8, the road shape estimating unit 42 estimates that the road shape is a downhill.

  Conventionally, it is possible to estimate the curvature of a road or the slope of a slope based on the yaw rate using a gyro sensor or the inclination of the vehicle using an acceleration sensor, but in any case, after the own vehicle enters a curve or a slope. Otherwise, accurate estimation was not possible. However, the image processing ECU 32 according to the present embodiment estimates whether or not the vehicle is on a curve or a slope on a stage before entering a curve or a slope on which the vehicle attitude has not changed without using a sensor. it can.

  As described above, the present invention has been described with reference to the above-described embodiments. However, the present invention is not limited to the above-described embodiments, and may be obtained by appropriately combining or replacing the configurations of the embodiments. It is included in the present invention. Further, it is also possible to appropriately change the combination and the order of processing in the embodiment based on the knowledge of those skilled in the art, and to add modifications such as various design changes to the embodiment. The embodiments described above can also be included in the scope of the present invention.

  C1 intersection, H1 horizontal line, L1 left white line, L1 'straight line, R1 rectangular area, C2 intersection, H2 horizontal line, L2 left white line, L2' straight line, R2 rectangular area, C3 intersection, H3 horizontal line, L3 left white line, L3 'straight line, R3 rectangular area, 10 vehicles, 12 headlight units, 14 control systems, 16 front monitoring cameras, 32 image processing ECUs, 34 light distribution control ECUs, 38 image information acquisition units, 40 white line detection units, 42 road shape estimation units, 44 Reference point calculation unit, 48 discrimination unit, 50 oncoming vehicle, 52 preceding vehicle, 110 vehicle lamp.

Claims (5)

A road shape estimating unit that determines a region including the roadway outer line from a captured image obtained by capturing an image in front of the vehicle, and estimates a road shape based on the shape of the region,
A reference point calculation unit that calculates a distant reference point based on an intersection of a straight line obtained by extending a diagonal line of the region and a horizontal line in the captured image,
An image processing apparatus comprising:   The image processing device according to claim 1, wherein the road shape estimating unit determines a rectangular region as the region, and estimates a road shape based on information correlated with an aspect ratio of the rectangular region. A determination unit configured to determine a situation ahead of the vehicle using the distant reference point,
The image processing apparatus according to claim 1, wherein the determination unit determines that a light spot moving obliquely downward from the distant reference point is an oncoming vehicle. A determination unit configured to determine a situation ahead of the vehicle using the distant reference point,
The image processing device according to claim 1, wherein the determination unit determines that a light spot that stays between the distant reference point and the host vehicle is a preceding vehicle. An image processing device according to claim 3 or 4,
A headlight unit that illuminates the front of the vehicle,
A light distribution control unit that controls light distribution of the headlight unit according to an attribute of a light spot determined by the image processing apparatus,
A vehicle lighting device comprising:

Images