CN113196740B - Image processing apparatus and method - Google Patents
Image processing apparatus and method Download PDFInfo
- Publication number
- CN113196740B CN113196740B CN201980083769.5A CN201980083769A CN113196740B CN 113196740 B CN113196740 B CN 113196740B CN 201980083769 A CN201980083769 A CN 201980083769A CN 113196740 B CN113196740 B CN 113196740B
- Authority
- CN
- China
- Prior art keywords
- light source
- source region
- unit
- luminance
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/16—Anti-collision systems
Abstract
The invention provides an image processing device capable of accurately identifying whether a target object is a headlight or a reflector regardless of the distance of a host vehicle, the brightness of a reflector, the headlight, and the like. An image acquisition unit (12) acquires an image from a camera (11), a luminance calculation unit (13) generates a luminance image, and a light source region setting unit (14) calculates a plurality of vehicle candidate regions each composed of luminance pixels equal to or greater than a predetermined reference value. A threshold value determination unit (15) determines a threshold value for each of a plurality of vehicle candidate regions based on irradiation direction information of the own headlamp from an irradiation direction information acquisition unit (17), and an object recognition unit (16) determines whether each region is a predetermined object for the vehicle candidate region. Information about the area as a light shielding object is added to the vehicle candidate area determined as the object, and the information is output to a light distribution control unit (18). For a vehicle candidate region determined to be not a predetermined object, information that is not a light shielding object is output to a light distribution control unit (18).
Description
Technical Field
The present invention relates to an image processing device for mounting on a vehicle.
Background
In the image processing apparatus described in patent document 1, an area having a luminance equal to or higher than a predetermined value is set as a light source area from an image. Then, a threshold value of the relation between the distance to the light source region and the brightness is used to determine whether it is a headlight or a reflector.
Further, patent document 1 describes the following: the high beam and low beam states of the vehicle are used, and the reflected light is made stronger at the high beam, so that the brightness threshold is made stricter, and the reflected light is made weaker at the low beam, so that the brightness threshold is made looser.
Prior art literature
Patent literature
Disclosure of Invention
Problems to be solved by the invention
However, in the technique described in patent document 1, if the correct distance to the host vehicle cannot be measured for the light source region detected on the image, or if the brightness of the reflection object is higher than expected due to the difference in the head lamps of the host vehicle, the reflection object may be erroneously detected as a head lamp or a rear lamp.
Further, in the technique described in patent document 1, when the brightness of the head lamp or the rear lamp is lower than expected, there is a possibility that the detection is not performed.
The purpose of the present invention is to realize an image processing device that can accurately identify whether a vehicle is a headlight or a reflector regardless of the distance between the vehicle and the reflector, the brightness of the headlight, or the like.
Technical means for solving the problems
In order to achieve the above object, the present invention is constructed as follows.
An image processing device is provided with: an image acquisition unit that acquires a captured image captured by an imaging device; a luminance calculating section that divides the captured image acquired by the image acquiring section into a plurality of pixel areas and calculates the luminance of each pixel area; a light source region setting unit that sets, as a light source region, a region including a plurality of the pixel regions whose luminances calculated by the luminance calculating unit are close to or greater than a set luminance; a threshold value determining unit configured to determine a threshold value for each of the plurality of light source regions set by the light source region setting unit; and an object recognition unit that recognizes an object using the threshold determined by the threshold determination unit.
ADVANTAGEOUS EFFECTS OF INVENTION
An image processing device capable of accurately identifying whether a target object is a headlight or a reflecting object irrespective of the distance between the vehicle and the brightness of the reflector, the headlight, or the like is realized.
Drawings
Fig. 1 is a schematic configuration diagram of a vehicle control system according to an embodiment of the present invention.
Fig. 2 is a flowchart of a light distribution control process of the light distribution control system of an embodiment of the present invention.
Fig. 3 is a flowchart of the vehicle identification process in the present embodiment.
Fig. 4 is a diagram illustrating brightness and area of a head lamp and a reflector.
Fig. 5 is a diagram showing an example of the variance value of the luminance of the object recognition unit.
Fig. 6 is an explanatory view of the light source region and the pixel region.
Fig. 7 is a diagram showing an example of a line-of-sight guide (outline).
Detailed Description
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
Examples
Fig. 1 is a schematic configuration diagram of a vehicle control system according to an embodiment of the present invention.
In fig. 1, a vehicle control system (may also be a light distribution control system) 100 is provided in a vehicle (sometimes referred to as a host vehicle) and includes: a camera (imaging device) 11 provided in front of the vehicle and periodically capturing images of a predetermined area in front of the vehicle in time series; an image processing device 1 that processes an image (captured image) captured by the camera 11; and a vehicle control device 2 that controls the vehicle based on the processing result of the image processing device 1.
The image processing apparatus 1 and the vehicle control apparatus 2 are computers (e.g., microcomputers), and each have an input unit (not shown), a central processing unit 3 (CPU or MPU) as a processor, a Read Only Memory (ROM) and a Random Access Memory (RAM) as a storage device 4, and an output unit (not shown).
The input unit (not shown) converts various information input to the image processing apparatus 1 and the vehicle control apparatus 2 to be capable of being operated by the CPU. The ROM is a recording medium in which a control program for executing an arithmetic process described later, various information necessary for executing the arithmetic process, and the like are appropriately stored, and the CPU performs predetermined arithmetic processes on signals received from the input unit, the ROM, and the RAM in accordance with the control program stored in the ROM.
Instructions for controlling the output object, information used for the output object, and the like are output from the respective output units of the image processing apparatus 1 and the vehicle control apparatus 2. The storage device 4 is not limited to the semiconductor memory such as the ROM and the RAM described above, and may be replaced with a magnetic storage device such as a hard disk drive.
The image processing apparatus 1 is a computer that performs an external recognition process including object detection based on a captured image (color image) of the camera 11, and includes: an image acquisition unit 12, a luminance calculation unit 13, a light source region setting unit 14, a threshold determination unit 15, an object identification unit 16, and an irradiation direction information acquisition unit 17.
The vehicle control device 2 is a computer that executes vehicle control processing based on the detection result of the object recognition unit 16 output from the image processing device 1, and includes a light distribution control unit 18. The image processing apparatus 1 and the vehicle control apparatus 2 in fig. 1 are composed of 2 different computers, but may be composed of an integrated computer.
The image acquisition section 12 performs the following processing: the camera 11 is controlled to periodically perform setting and photographing of exposure time (shutter speed), acquire a time series of color images, and store them in the image processing apparatus 1. The color image is defined by an RGB color model, and the colors of the pixels constituting the color image are defined by a combination of the brightnesses of red (R), green (G), and blue (B). In this embodiment, the respective luminances of R, G, B are expressed by integer values (i.e., 256 gray levels) of 0 to 255, and the colors of the pixels are defined by a combination of 3 values (R value, G value, and B value). The color image is captured at a prescribed period, for example, 30 frames per second.
The luminance calculating unit 13 changes and determines the luminance calculation formula based on the luminance index value located in a predetermined area in the center of the luminance image generated in the past (for example, before a predetermined frame), the output of the object identifying unit 16 including the information of the predetermined object to be detected by the object identifying unit 16, and the output of the vehicle control device 2 including the information of the application program executed by the vehicle control device 2. This is to adjust a constant or the like of the luminance calculation formula to clarify an image when a situation of the vehicle outside rapidly changes, and to output information of a predetermined object as a detection object or the like from the object recognition unit 16 to the luminance calculation unit 13.
Each luminance calculation formula is defined by a sum of values obtained by multiplying R, G, and B values by predetermined coefficients, and the luminance calculation unit 13 calculates the luminance of a pixel constituting a color image by substituting the R, G, and B values into the formula. The brightness is calculated for each pixel of the captured image of the camera 11, and the image in which the brightness is collected is made a brightness image.
The light source region setting unit 14 detects a pixel region 41 having a luminance equal to or higher than a set threshold (as shown in fig. 6, which is an explanatory diagram of a light source region and a pixel region) from the luminance image outputted from the luminance calculating unit 13, and integrates the pixels having a luminance equal to or higher than the set threshold and the adjacent pixel regions 41 as a vehicle candidate region (light source region) 42 (as shown in fig. 6), as will be described later.
Then, the variance value is calculated as a value representing the area of the vehicle candidate region, the maximum luminance, and the variance of the luminance, and a process of storing them in the storage device 4 in the image processing device 1 is performed. The value indicating the deviation of the luminance is not limited to the variance value, and may be replaced with, for example, a standard deviation, and when the variance value or the standard deviation is equal to or greater than a set variance value or a set standard deviation, the light source is recognized as a reflection object.
The threshold value determining unit 15 of the present embodiment sets a threshold value for each light source region based on the information of the light source region as the output of the light source region setting unit 14 and the irradiation direction information of the own headlamp as the output of the irradiation direction information acquiring unit 17. Therefore, the threshold value can be changed for a light source region or the like that appears at the time when the own headlamp is switched from low beam to high beam.
The switching of the host vehicle headlamp of which the threshold is changed is not limited to the timing of switching from the low beam to the high beam, and may be, for example, instead of the timing of switching from the high beam to the low beam.
The central processing unit 3 of the image processing apparatus 1 of the present embodiment includes: an irradiation direction information acquisition unit 17 that acquires information on the irradiation direction of the headlight of the host vehicle; a camera (image pickup device) 11; an image acquisition unit 12 that acquires a captured image of the camera 11; a luminance calculating section 13 that divides the image acquired by the image acquiring section 12 into a plurality of pixel regions 41 and calculates the luminance of each pixel region 41; a light source region setting unit 14 that sets, as a light source region 42, a region including a plurality of pixel regions 41 that are close to each other and whose luminance calculated by the luminance calculating unit 13 is equal to or higher than a set luminance (for example, a luminance of a headlight of 600m or higher can be detected); a threshold value determining unit 15 that determines a threshold value for each light source region 42 based on the plurality of light source regions 42 set by the light source region setting unit 14 and the irradiation direction of the headlamp acquired by the irradiation direction information acquiring unit 17; and an object recognition unit 16 that recognizes the object using the threshold determined by the threshold determination unit 15.
The threshold value determining unit 15 of the central processing unit 3 of the image processing apparatus 1 of the present embodiment detects that the headlight is switched from low beam to high beam based on the information of the irradiation direction, and changes the threshold value when the object is recognized by the object recognizing unit 16 for each light source region 42 set by the light source region setting unit 14.
The object recognition unit 16 of the present embodiment performs a process of recognizing a predetermined object in a vehicle candidate region by using the color image acquired by the image acquisition unit 12 and the output of the light source region setting unit 14, and outputs the recognition result to the vehicle control device 2. The "predetermined object" to be identified by the object identification unit 16 may be determined according to the type of the application (program) executed by the vehicle control device 2.
The central processing device 3 of the image processing device 1 of the present embodiment calculates the area of the light source region 42 and the maximum luminance in the plurality of pixel regions 41 included in the light source region 42 in the light source region setting section 14. Then, the object recognition unit 16 recognizes whether the light source region 42 set by the light source region setting unit 14 is an object or a reflection object, using the threshold value determined by the threshold value determination unit 15.
In the object recognition unit 16, when the light source area is the same in size as the reflective object such as a headlight, a sign other than a backlight, or a sign, which cannot be recognized due to the relationship between the luminance and the distance, the reflective object is the reflection of the headlight of the vehicle, and the self-luminous headlight and the backlight are brighter and have higher luminance. Therefore, by comparing the brightest portion in the light source region with the size of the light source region, a large reflector such as a sign or a sign can be recognized as an object other than the head lamp or the rear lamp.
In the central processing unit 3 of the image processing apparatus 1 of the present embodiment, the light source region setting unit 14 sets the object region and the reflection object region based on the luminance distribution of the plurality of pixel regions 41 included in the light source region 42, and the object recognition unit 16 recognizes whether the light source region 42 set by the light source region setting unit 14 is the object region or the reflection object region using the threshold value determined by the threshold value determination unit 15.
A light source region 42 of a small reflector (a diagram illustrating the brightness and area of the head lamp and the reflector, that is, a mark 21B of an image 21 of fig. 4 a) such as a line-of-sight guide (outline) 51 (shown in fig. 7, which is an example of the line-of-sight guide (outline)) has a small rectangular area, and in the maximum brightness distribution 22 shown in fig. 4B, the head lamp 21a (fig. 4 a)), the rear lamp (circular mark) and the mark (x mark) overlap, and therefore, the head lamp, the rear lamp and the mark other than them cannot be completely recognized due to the relation of the brightness and the size.
However, fig. 5 is a diagram showing an example of the variance value of the brightness of the object recognition unit.
Since the head lamp and the rear lamp are self-luminous objects as shown in fig. 5 (a), (B), and (C), the luminance deviation in the light source region 42 is large as shown in the head lamp luminance distribution 31, and the line-of-sight guiding mark (outline mark) 51 has a characteristic of small luminance deviation as shown in the outline mark luminance distribution 32 due to reflection. Therefore, as shown in the graph 33 of fig. 5C, by comparing the luminance distribution of the light source region 42 showing the relationship between the variance value and the distance and separating the light source region by setting the threshold 34, it is possible to identify a small reflector such as the line-of-sight guiding mark (outline mark) 51 as an object other than the head lamp or the rear lamp.
The central processing unit 3 of the image processing apparatus 1 of the present embodiment includes: an irradiation direction information acquisition unit 17 that acquires information on the irradiation direction of the headlight of the host vehicle; an imaging device 11; an image acquisition unit 12 that acquires a captured image of the imaging device 11; a luminance calculating section 13 that divides the image acquired by the image acquiring section 12 into a plurality of regions 41 and calculates the luminance of each pixel region 41; a light source region setting unit 14 that sets, as a light source region 42, a region including a plurality of pixel regions 41 that are close to each other and whose luminance calculated by the luminance calculating unit 13 is equal to or higher than a set luminance; a threshold value determining unit 15 that determines a threshold value for each light source region 42 based on the plurality of light source regions 42 set by the light source region setting unit 14 and the irradiation direction of the headlamp acquired by the irradiation direction information acquiring unit 17; and an object recognition unit 16 that recognizes whether the light source located in the light source region 42 is an object or a reflection object, using the threshold value determined by the threshold value determination unit 15, based on the respective areas of the plurality of light source regions 42 and the luminance of the respective pixel regions 41 of the plurality of light source regions 42.
By changing the threshold value for each light source region by the threshold value determining unit 15 based on the information of the irradiation direction of the head lamp of the vehicle, for example, setting a strict threshold value for the light source region appearing at the time of the change in the irradiation direction of the head lamp, setting a low threshold value for the other light source regions, and the like, the degree of freedom can be recognized more than by recognizing all the light source regions by a predetermined threshold value.
As a result, the light source and the reflector can be identified. Further, the object recognition unit 16 recognizes whether the object is an object or a reflecting plate by using the relationship between the brightness and the area of the light source region using the changed threshold value.
The central processing unit 3 of the image processing apparatus 1 of the present embodiment includes: an irradiation direction information acquisition unit 17 that acquires information on the irradiation direction of the headlight of the host vehicle; an imaging device 11; an image acquisition unit 12 that acquires a captured image of the imaging device 11; a luminance calculating section 13 that divides the image acquired by the image acquiring section 12 into a plurality of pixel regions 41 and calculates the luminance of each pixel region 41; a light source region setting unit 14 that sets, as a light source region 42, a region including a plurality of pixel regions 41 that are close to each other and whose luminance calculated by the luminance calculating unit 13 is equal to or higher than a set luminance; a threshold value determining unit 15 that determines a threshold value for each light source region 42 based on the plurality of light source regions 42 set by the light source region setting unit 14 and the irradiation direction of the headlamp acquired by the irradiation direction information acquiring unit 17; and an object recognition unit 16 that recognizes whether the light source located in the light source region 42 is an object or a reflection object, using the threshold value determined by the threshold value determination unit 15, based on the distribution of the luminance of each pixel region 41 included in the light source region 42 set by the light source region setting unit 14.
By changing the threshold value for each light source region by the threshold value determining unit 15 based on the information of the irradiation direction of the head lamp of the vehicle, for example, setting a strict threshold value for the light source region appearing at the time of the change in the irradiation direction of the head lamp, setting a low threshold value for the other light source regions, and the like, the degree of freedom can be recognized more than by recognizing all the light source regions by a predetermined threshold value. As a result, the light source and the reflection can be identified. Further, the object recognition unit 16 recognizes whether the object is an object or a reflecting plate by using the changed threshold value and the distribution of the brightness of the light source region.
The central processing unit 3 of the image processing apparatus 1 of the present embodiment includes: an irradiation direction information acquisition unit 17 that acquires information on the irradiation direction of the headlight of the host vehicle; an imaging device 11; an image acquisition unit 12 that acquires a captured image of the imaging device 11; a luminance calculating section 13 that divides the image acquired by the image acquiring section 12 into a plurality of pixel regions 41 and calculates the luminance of each pixel region 41; a light source region setting unit 14 that sets, as a light source region 42, a region including a plurality of pixel regions 41 that are close to each other and whose luminance calculated by the luminance calculating unit 13 is equal to or higher than a set luminance; a threshold value determining unit 15 that determines a threshold value for each light source region 42 based on the plurality of light source regions 42 set by the light source region setting unit 14 and the irradiation direction of the headlamp acquired by the irradiation direction information acquiring unit 17; and an object recognition unit 16 that recognizes whether the light source located in the light source region 42 is an object or a reflection object, using the threshold value determined by the threshold value determination unit 15, based on the areas of the plurality of light source regions 42, the brightness of each pixel region 41 of the plurality of light source regions 42, and the distribution of the brightness of each pixel region 41 included in the light source region 42 set by the light source region setting unit 14.
By changing the threshold value for each light source region by the threshold value determining unit 15 based on the information of the irradiation direction of the head lamp of the vehicle, for example, setting a strict threshold value for the light source region appearing at the time of the change in the irradiation direction of the head lamp, setting a low threshold value for the other light source regions, and the like, it is possible to perform recognition with a higher degree of freedom than recognizing all the light source regions by a predetermined threshold value, and as a result, it is possible to recognize the light source and the reflecting plate. Further, the object recognition unit 16 can recognize whether the object is an object or a reflecting plate by using the relationship between the brightness and the area of the light source region and the distribution of the brightness of the light source region by using the changed threshold value.
In the central processing unit 3 of the image processing apparatus 1 of the present embodiment, the object recognition unit 16 recognizes whether the light source located in the light source region 42 is an object or a reflection object based on the areas of the plurality of light source regions 42, the luminances of the pixel regions 41 of the plurality of light source regions 42, and the distribution of the luminances of the pixel regions 41 included in the light source region 42 set by the light source region setting unit 14. Since the reflective object such as a sign or a sign is a reflection of the headlight of the vehicle, there is a tendency that brightness indicating brightness on an image is lower than that of the headlight or the backlight. Thus, the head lamp and the sign or sign of the rectangular area of the same light source region can be separated according to the difference in brightness. However, a small reflector such as a line-of-sight guide (outline) is smaller than the headlight, and is difficult to recognize from the relationship between the area and the luminance. Accordingly, focusing on the variation in luminance of the light source region, the variation in the front and rear lamps as self-luminous objects from bright places to dark places tends to be large, and the line-of-sight guiding marks (outline marks) as reflecting objects tend to be small, and the variation in luminance can be used for recognition. Therefore, by combining the relationship between the area and the luminance and the relationship between the luminance deviation, false detection countermeasures for the sign, and the line-of-sight guide (outline sign) can be performed.
The object recognition unit 16 of the central processing unit 3 of the image processing apparatus 1 according to the present embodiment determines the light source region 42 as the object when the ratio of the maximum luminance of the pixel region 41 of each of the plurality of light source regions 42 to the area of each of the plurality of light source regions 42 set by the light source region setting unit 14 is equal to or greater than the set threshold value 23.
The brightness and the size of the head lamp and the back lamp are standardized (notice < second section > item 120 (head lamp, etc.) of a specific item for determining a safety standard of a road transportation vehicle), and it is expected that the head lamp and the back lamp are displayed with a certain brightness and size on an image according to the detected distance. Therefore, when the ratio of the brightness to the size on the image is equal to or greater than a predetermined value, the object can be determined. The setting of the threshold 23 (fig. 4 (B)) is not limited to the maximum luminance/rectangular area=1, and may be, for example, the maximum luminance/rectangular area=0.5.
The object recognition unit 16 of the central processing unit 3 of the image processing apparatus 1 of the present embodiment determines the light source region 42 as a reflection object when the ratio of the maximum brightness of the pixel region 41 of each of the plurality of light source regions 42 to the area of each of the plurality of light source regions 42 set by the light source region setting unit 14 is smaller than the set threshold 23.
Since the reflector such as a sign or a sign is a reflection of the headlight of the host vehicle, if the comparison is made with the rectangular area of the light source region 42 being the same, the brightness on the image is darker than the headlight. Therefore, when the ratio of the brightness to the size is smaller than the set threshold 23, it can be determined as a reflection object.
The object recognition unit 16 of the central processing unit 3 of the image processing apparatus 1 according to the present embodiment determines the light source region 42 as the object when the variance value or standard deviation value of the luminance of each pixel region 41 of the light source region 42 set by the light source region setting unit 14 is equal to or greater than the set threshold 34.
Since the head lamp and the rear lamp are self-luminous objects, the luminance deviation in the light source region is larger than that of the reflecting object such as the line-of-sight guide mark (outline mark) 51 (see the head lamp luminance distribution 31 of fig. 5 a). Therefore, when the variance value or standard deviation of the luminance is equal to or greater than the set threshold 34, it can be determined as the object.
The object recognition unit 16 of the central processing unit 3 of the image processing apparatus 1 of the present embodiment determines the light source region 42 as a reflection object when the variance value or standard deviation of the luminance of each pixel region 41 of the light source region 42 set by the light source region setting unit 14 is smaller than the set threshold 34.
The reflection object such as the line-of-sight guide (outline) 51 has a smaller variation in brightness (outline brightness distribution 32) in the light source region than the headlamp. Therefore, when the variance value or standard deviation of the luminance is smaller than the set threshold 34, it can be recognized as a reflection object.
The light distribution control system 100 of the present embodiment includes: an image processing apparatus 1 and a light distribution control unit 18, wherein the image processing apparatus 1 includes: an irradiation direction information acquisition unit 17 that acquires information on the irradiation direction of the headlight of the host vehicle; an imaging device 11; an image acquisition unit 12 that acquires a captured image of the imaging device 11; a luminance calculating section 13 that divides the image acquired by the image acquiring section 12 into a plurality of pixel regions 41 and calculates the luminance of each pixel region 41; a light source region setting unit 14 that sets, as a light source region 42, a region including a plurality of pixel regions 41 that are close to each other and whose luminance calculated by the luminance calculating unit 13 is equal to or higher than a set luminance; a threshold value determining unit 15 that determines a threshold value for each light source region 42 based on the plurality of light source regions 42 set by the light source region setting unit 14 and the irradiation direction of the headlamp acquired by the irradiation direction information acquiring unit 17; and an object recognition unit 16 that recognizes whether or not the light source located in the light source region 42 is an object, based on the areas of the plurality of light source regions 42, the brightness of each pixel region 41 of the plurality of light source regions 42, and the brightness distribution of each pixel region 41 included in the light source region 42 set by the light source region setting unit 14, using the threshold determined by the threshold determination unit 15, and the light distribution control unit 18 performs light distribution control based on the light shielding region of the headlamp determined based on the recognition result of the object recognition unit 16.
The light distribution control unit 18 of the vehicle control device 2 executes light distribution control (light distribution control application) of the own headlamp based on the recognition result (vehicle detection result) of the object recognition unit 16 of the image processing device 1. As the light distribution control, for example, a light distribution control is performed in which the own headlight is set to a high beam when there is no preceding vehicle or no oncoming vehicle in front of the own headlight, and the own headlight is set to a low beam when there is a preceding vehicle or a oncoming vehicle.
When another vehicle is detected in front of the host vehicle, a shade region of the host vehicle headlamp is determined so as to exclude the other vehicle from the irradiation region of the host vehicle headlamp, and the light distribution control of the host vehicle headlamp is performed based on the shade region. In the latter case, an actuator (e.g., a plurality of motors) for changing the optical axes of the plurality of headlamps and a light shielding mechanism for shielding a part of the light of each of the headlamps are provided in the host vehicle, and control signals for these are output from the light distribution control unit 18.
Next, fig. 2 is a flowchart of the light distribution control process of the light distribution control system 100. The flow of the light distribution control process will be described with reference to fig. 2. The light distribution control system 100 (the image processing apparatus 1 and the vehicle control apparatus 2) repeatedly executes the flowchart of fig. 2 at predetermined control cycles.
In fig. 2, when the processing is started, the image acquisition unit 12 sets the exposure time of the image to be captured next to the camera 11, captures the image at a predetermined timing, and acquires a color image (captured image) in step S11.
Next, in step S12, the luminance calculating unit 13 generates a luminance image according to the luminance calculation formula.
Next, in step S13, the light source region setting unit 14 calculates a plurality of vehicle candidate regions (light source regions 42) each including luminance pixels (pixel regions 41) equal to or greater than a predetermined reference value for the luminance image generated in step S12.
Next, in step S14, the threshold value determining unit 15 determines a threshold value for each of the plurality of vehicle candidate regions (light source regions 42) calculated in step S13 based on the irradiation direction information of the own headlamp from the irradiation direction information acquiring unit 17.
Next, in step S15, the object recognition unit 16 determines whether or not each of the plurality of vehicle candidate regions (light source regions 42) calculated in step S13 is a predetermined object (object recognition process (vehicle recognition process)) based on the threshold value determined in step S14.
Here, the vehicle identification process in step 15 will be described.
Fig. 3 is a flowchart of the vehicle identification process in the present embodiment.
In fig. 3, when the object recognition unit 16 starts the vehicle recognition process, in step S21, it is determined whether or not the maximum brightness is equal to or lower than the threshold value, and thus, it is determined whether or not the filtering process is performed based on the maximum brightness. In step S21, if the maximum luminance is not below the threshold, the process proceeds to step S25, and if the maximum luminance is below the threshold, the process proceeds to step S22.
Next, in step S22, it is checked whether the distance to the host vehicle is an improper value, whether the size (size) of the vehicle candidate area is equal to or larger than a certain value, whether the actual size obtained from the distance and the size of the vehicle candidate area is excessively large, whether the lateral position is excessively far from the lane, or the like, by comparison with the threshold value, and if there is no problem, the process proceeds to the next step S23. If there is a problem in step S22, the flow proceeds to step S26, and the process proceeds to step S26, where the process is performed except for a predetermined object (excluding the object as a false detection object).
In step S23, it is determined whether or not the ratio of the area of the vehicle candidate region to the maximum luminance is equal to or greater than a threshold value, and if the ratio of the area of the vehicle candidate region to the maximum luminance is equal to or greater than the threshold value, it is determined that the vehicle candidate region is a predetermined object, and the process proceeds to step S24.
In step S23, when the ratio is smaller than the threshold value, the routine proceeds to step S26, where it is determined that the object is other than the predetermined object.
In step S24, it is determined whether or not the variance value of the luminance of the vehicle candidate region is equal to or greater than a threshold value, and if the variance value is equal to or greater than the threshold value, it is determined that the vehicle candidate region is a predetermined object, and the process proceeds to step S25. If the threshold value is smaller than 34 in step S24, the routine proceeds to step S26, where it is determined that the predetermined object is not present.
In step S25, the vehicle candidate regions having passed through steps S22, S23, and S24 are stored as a group as detection targets.
Next, in step S27, the vehicle candidate region equal to or greater than the threshold value is identified as a predetermined object based on the stored luminance threshold value for each distance of the vehicle candidate region.
The above is a description of the vehicle identification process in step 15.
In step S15, information about a vehicle candidate region determined to be a predetermined object, which is a light shielding target, is added to the vehicle candidate region, and the information is output to the light distribution control unit 18. In addition, information that is not a light shielding object is added to the vehicle candidate region determined to be not the predetermined object in step S15, and the vehicle candidate region is output to the light distribution control unit 18, and the flow proceeds to step S16.
In step S16, the light distribution control unit 18 determines a shade area of the own headlamp based on the output information of the object recognition unit 16, and performs light distribution control of the own headlamp so that the shade area is excluded from the shade area of the own headlamp.
Accordingly, 1 cycle of the light distribution control process of the light distribution control system 100 is completed. The light distribution control system executes a series of processes again from step S11 only after waiting for a predetermined control period.
As described above, according to one embodiment of the present invention, by changing the threshold value for the light source region detected at the timing of switching the irradiation direction of the head lamp of the host vehicle, a different threshold value is set for each light source region. For example, when the light source detected at the time of switching from low beam to high beam is highly likely to be a reflecting object, the threshold value used in the subsequent recognition processing is strictly set, so that the head lamp, the rear lamp, and the reflecting object can be recognized.
In addition, by setting different thresholds for the respective light source regions, undetected detection of the head lamp and the rear lamp can be suppressed.
In the recognition processing, since the head lamp and the rear lamp are self-luminous and the reflecting object such as a sign or a sign is a reflection of the head lamp of the vehicle, the head lamp and the rear lamp are brightly displayed on the image when the light source areas on the image are the same size. Therefore, the head lamp, the rear lamp, and the reflector can be classified according to the ratio of the size and the brightness on the image.
In addition, the line-of-sight guide marks (outline marks) which are smaller in size than the head lamp or the rear lamp and cannot be classified by the ratio of the size to the brightness are smaller in the brightness variation on the image than the head lamp or the rear lamp. Therefore, by using the variance value or standard deviation value of the luminance of the light source region, the head lamp, the rear lamp, and the outline marker can be classified.
In the above example, the threshold value determination unit 15 determines the threshold value in consideration of the irradiation direction information of the headlight of the vehicle supplied from the irradiation direction information acquisition unit 17, but the threshold value may be determined based on only the information of the light source region from the light source region setting unit 14.
Symbol description
1 … image processing apparatus, 2 … vehicle control apparatus; 3 … Central processing Unit, 4 … storage Unit, 11 … video camera (image pickup apparatus), 12 … image acquisition unit, 13 … luminance calculation unit, 14 … light source region setting unit, 15 … threshold determination unit, 16 … object identification unit, 17 … irradiation direction information acquisition unit, 18 … light distribution control unit, 21 … image, 21a … headlamp, 21b … logo, 22 … maximum luminance distribution, 23, 34 … set threshold, 31 … headlamp luminance distribution, 32 … outline target luminance distribution, 41 … pixel region, 42 … light source region, 51 … line-of-sight guide (outline target), 100 … vehicle control System (light distribution control System).
Claims (7)
1. An image processing apparatus, comprising:
an image acquisition unit that acquires a captured image captured by an imaging device;
a luminance calculating unit that divides the captured image acquired by the image acquiring unit into a plurality of pixel areas and calculates the luminance of each pixel area;
a light source region setting unit that sets, as a light source region, a region including a plurality of pixel regions whose luminances calculated by the luminance calculating unit are close to or greater than a set luminance;
a threshold value determining unit configured to determine a threshold value for each of the plurality of light source regions set by the light source region setting unit; and
an object identification unit that identifies an object using the threshold determined by the threshold determination unit,
the light source region setting section calculates a region of the light source region and a maximum luminance in the plurality of pixel regions included in the light source region, and sets a variance value or a standard deviation value of the luminance of each of the pixel regions included in the light source region,
in the object recognition unit, a false detection countermeasure is performed using a relationship between the area and the maximum luminance and a relationship between the variance value or the standard deviation value and a distance from the light source region to the host vehicle.
2. The image processing apparatus according to claim 1, wherein,
the vehicle headlamp irradiation system further includes an irradiation direction information acquisition unit that acquires information on an irradiation direction of the vehicle headlamp, and the threshold determination unit determines a threshold value for each of the plurality of light source regions set by the light source region setting unit based on the information on the irradiation direction of the headlamp acquired by the irradiation direction information acquisition unit.
3. The image processing apparatus according to claim 2, wherein,
the threshold value determining unit detects that the headlight is switched from low beam to high beam based on the information of the irradiation direction, and changes the threshold value for identifying the object in the object identifying unit for each of the light source regions set by the light source region setting unit.
4. The image processing apparatus according to claim 1, wherein,
the object recognition unit determines the light source region as the object when a ratio of a maximum luminance of each of the plurality of light source regions to an area of each of the plurality of light source regions set by the light source region setting unit is equal to or greater than a set threshold.
5. The image processing apparatus according to claim 1, wherein,
the object recognition unit determines the light source region as a reflection object when a ratio of a maximum luminance of each of the plurality of light source regions to an area of each of the plurality of light source regions set by the light source region setting unit is smaller than a set threshold.
6. The image processing apparatus according to claim 1, wherein,
the object recognition unit recognizes the light source located in the light source region as the object when the variance value or the standard deviation value of the luminance of each of the regions included in the light source region set by the light source region setting unit is equal to or larger than a set variance value or a set standard deviation value.
7. The image processing apparatus according to claim 1, wherein,
the object recognition unit determines the light source region as a reflection object when the variance value or the standard deviation value of the luminance of each of the regions included in the light source region set by the light source region setting unit is smaller than a set variance value or a set standard deviation value.
Applications Claiming Priority (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP2018236751 | 2018-12-18 | ||
| JP2018-236751 | 2018-12-18 | ||
| PCT/JP2019/045357 WO2020129522A1 (en) | 2018-12-18 | 2019-11-20 | Image processing device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113196740A CN113196740A (en) | 2021-07-30 |
| CN113196740B true CN113196740B (en) | 2023-07-04 |
Family
ID=71101259
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201980083769.5A Active CN113196740B (en) | 2018-12-18 | 2019-11-20 | Image processing apparatus and method |
Country Status (3)
| Country | Link |
|---|---|
| JP (1) | JP7241772B2 (en) |
| CN (1) | CN113196740B (en) |
| WO (1) | WO2020129522A1 (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022264894A1 (en) * | 2021-06-17 | 2022-12-22 | ヌヴォトンテクノロジージャパン株式会社 | Object detecting device, object detecting method, and program |
| CN117480409A (en) * | 2021-06-17 | 2024-01-30 | 新唐科技日本株式会社 | Object detection device, object detection method, and program |
| CN113650553A (en) * | 2021-09-17 | 2021-11-16 | 北京航空航天大学 | Intelligent adjusting control device and method for automobile headlamp |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2011025894A (en) * | 2009-07-29 | 2011-02-10 | Toyota Central R&D Labs Inc | Vehicle lamp recognizing device and program |
| CN103348394A (en) * | 2011-04-13 | 2013-10-09 | 日产自动车株式会社 | Driving assistance device and adjacent vehicle detection method therefor |
| CN106295494A (en) * | 2015-06-26 | 2017-01-04 | 富士重工业株式会社 | Exterior environment recognition device |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPS63184900A (en) * | 1987-01-27 | 1988-07-30 | 松下電器産業株式会社 | Vehicle detector |
| US6049171A (en) * | 1998-09-18 | 2000-04-11 | Gentex Corporation | Continuously variable headlamp control |
| EP2698777B1 (en) * | 2011-04-13 | 2016-04-13 | Toyota Jidosha Kabushiki Kaisha | Vehicle-mounted peripheral object recognition device and driving assistance device using same |
| DE102011085689A1 (en) * | 2011-11-03 | 2013-05-08 | Robert Bosch Gmbh | Method and device for determining a group from at least two adjacently arranged lighting units while driving a vehicle |
| JP5820843B2 (en) * | 2013-05-29 | 2015-11-24 | 富士重工業株式会社 | Ambient environment judgment device |
| JP6227898B2 (en) * | 2013-05-29 | 2017-11-08 | 株式会社Ｓｕｂａｒｕ | Image processing device |
| JP6310899B2 (en) * | 2015-11-25 | 2018-04-11 | 株式会社Ｓｕｂａｒｕ | Outside environment recognition device |
| JP6200481B2 (en) * | 2015-11-25 | 2017-09-20 | 株式会社Ｓｕｂａｒｕ | Outside environment recognition device |
-
2019
- 2019-11-20 CN CN201980083769.5A patent/CN113196740B/en active Active
- 2019-11-20 JP JP2020561236A patent/JP7241772B2/en active Active
- 2019-11-20 WO PCT/JP2019/045357 patent/WO2020129522A1/en active Application Filing
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2011025894A (en) * | 2009-07-29 | 2011-02-10 | Toyota Central R&D Labs Inc | Vehicle lamp recognizing device and program |
| CN103348394A (en) * | 2011-04-13 | 2013-10-09 | 日产自动车株式会社 | Driving assistance device and adjacent vehicle detection method therefor |
| CN106295494A (en) * | 2015-06-26 | 2017-01-04 | 富士重工业株式会社 | Exterior environment recognition device |
Also Published As
| Publication number | Publication date |
|---|---|
| JP7241772B2 (en) | 2023-03-17 |
| WO2020129522A1 (en) | 2020-06-25 |
| CN113196740A (en) | 2021-07-30 |
| JPWO2020129522A1 (en) | 2021-10-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113196740B (en) | Image processing apparatus and method | |
| CN108716983B (en) | Optical element detection method and device, electronic equipment, storage medium | |
| EP2448251B1 (en) | Bundling night vision and other driver assistance systems (DAS) using near infra red (NIR) illumination and a rolling shutter | |
| US7639878B2 (en) | Shadow detection in images | |
| US8503818B2 (en) | Eye defect detection in international standards organization images | |
| JP5846872B2 (en) | Image processing device | |
| US7986812B2 (en) | On-vehicle camera with two or more angles of view | |
| US20090278922A1 (en) | Image sensor with integrated region of interest calculation for iris capture, autofocus, and gain control | |
| US11830256B2 (en) | Systems and methods for locating a retroreflective object in a digital image | |
| WO2020059565A1 (en) | Depth acquisition device, depth acquisition method and program | |
| IES20070821A2 (en) | Face tracking in a camera processor | |
| US10121083B2 (en) | Vehicle exterior environment recognition apparatus | |
| CN108716982A (en) | Optical element detection method, device, electronic equipment and storage medium | |
| US20150054955A1 (en) | Image processing method of vehicle camera and image processing apparatus using the same | |
| KR20210006276A (en) | Image processing method for flicker mitigation | |
| KR20160061132A (en) | Apparatus and method for recognizing the number of cars and a computer-readable recording program for performing the said method, the recording medium | |
| CN108600740A (en) | Optical element detection method, device, electronic equipment and storage medium | |
| KR101606476B1 (en) | Apparatus and method for recognition signal light using multiple exposure image of camera | |
| CN109287110B (en) | Substrate position recognition device, position recognition processing device, and substrate manufacturing method | |
| US20190045100A1 (en) | Image processing device, method, and program | |
| US11240439B2 (en) | Electronic apparatus and image capture apparatus capable of detecting halation, method of controlling electronic apparatus, method of controlling image capture apparatus, and storage medium | |
| CN108760245B (en) | Optical element detection method and device, electronic equipment, readable storage medium storing program for executing | |
| JP2007124676A (en) | On-vehicle image processor | |
| JP2010286995A (en) | Image processing system for vehicle | |
| JP2010134363A (en) | Illumination control device and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |