JPWO2019142586A1 - Image processing system and light distribution control system - Google Patents

Abstract

The present invention provides an image processing system capable of identifying an object to be detected with high accuracy without increasing the cost. The present invention includes an image pickup device 11, a brightness calculation unit 14 that generates a brightness image from an image captured by the image pickup device using a brightness calculation formula, and a detection unit 15 that detects a predetermined object based on the brightness image. An image processing system including the image processing device 1 having the image processing device 1, wherein the image processing device 1 is a luminance calculation unit according to a luminance index value of a predetermined region in a luminance image or a type of a predetermined object detected by the detection unit. Further, a luminance calculation formula changing unit 13 for changing the luminance calculation formula used in the above is provided.

Description

The present invention relates to an image processing system mounted on a vehicle and a light distribution control system that controls the light distribution of vehicle headlights.

Inventions relating to an image processing system for mounting on a vehicle have been conventionally known (see Patent Document 1 below). The image processing system described in Patent Document 1 includes an image pickup means mounted on a vehicle and an image analysis means for acquiring an image taken by the image pickup means and analyzing the image. This image analysis means analyzes the image taken by the image pickup means and detects the position of the tail light of the preceding vehicle that glows red.

In this image processing system, first, red pixels are detected from the captured image, and adjacent red pixels are connected to create an "element group" (= light spot). Next, the vehicle (tail light) and other objects based on the features of the element group (for example, the size of the rectangle inscribed or circumscribed in the element group, the maximum / minimum brightness of the pixels included in the element group, etc.). (Reflector, etc.) is identified.

Japanese Unexamined Patent Publication No. 2014-232431

The image processing system of Patent Document 1 has a problem of erroneously detecting an object having a color close to red, such as an orange reflector on a road surface, as a tail light of a preceding vehicle.

In order to improve the accuracy of tail light detection, it is preferable to capture an image under exposure conditions suitable for the process of determining the color and brightness of an object. However, when a plurality of recognition functions are operated simultaneously on one system, a high-cost camera and a circuit are required to take an image under the optimum exposure conditions for each function. On the other hand, in order to reduce the cost, there is no choice but to share the exposure conditions with multiple recognition functions and input an image with brightness that is not always appropriate from the viewpoint of tail light detection to perform tail light detection processing. There is. The same problem can occur not only when the detection target is the tail light of another vehicle but also when detecting a self-luminous light emitter such as a headlight or a traffic light of an oncoming vehicle. Further, the same problem may occur not only in the light emitting body but also in the case where blackout or whiteout occurs in the captured image at the entrance / exit of the tunnel when detecting another vehicle in the daytime.

The present invention has been made in view of the above problems, and an object of the present invention is to provide an image processing system capable of identifying an object to be detected with high accuracy without increasing the cost.

The present application includes a plurality of means for solving the above problems. For example, an image pickup device, a brightness calculation unit that generates a brightness image from an image captured by the image pickup device by using a brightness calculation formula, and a brightness calculation unit. An image processing system including an image processing device having a detection unit that detects a predetermined object based on the luminance image, and the image processing apparatus is a luminance index value of a predetermined region in the luminance image or the above. It is characterized by further including a luminance calculation formula changing unit that changes the luminance calculation formula used in the luminance calculation unit according to the type of the predetermined object detected by the detection unit.

According to the present invention, the detection rate of the detection target can be improved without increasing the cost even in a situation where an image captured under exposure conditions suitable for detecting the detection target cannot be obtained.

The schematic block diagram of the vehicle control system 100. The explanatory view of the predetermined area (front vehicle area) A set on the luminance image. The flowchart of the light distribution control process by the vehicle control system 100. The flowchart of the determination process (change process) of the luminance calculation formula by the luminance calculation formula change part 13. An example of an image of the tail light and reflector of the preceding vehicle. The figure which showed the relationship between the distance and the luminance value about the pixel of the image 51, 52 which concerns on a tail light and a reflector under the same exposure condition. The figure which shows an example of the luminance image and the luminance distribution of the orange reflector on the road. An example of a flowchart of a luminance calculation formula determination process (change process) by the luminance calculation formula changing unit 13. An example of a flowchart of a luminance calculation formula determination process (change process) by the luminance calculation formula changing unit 13. An example of a flowchart of a luminance calculation formula determination process (change process) by the luminance calculation formula changing unit 13. A table summarizing the contents of each brightness calculation formula and the conditions under which each brightness calculation formula is selected.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is a schematic configuration diagram of a vehicle control system according to the present embodiment. The vehicle control system 100 in this figure is provided in a vehicle (sometimes referred to as a own vehicle), and is installed in front of the vehicle as an imaging device that periodically captures images of a predetermined area in front of the vehicle in chronological order. The camera 11 is provided with an image processing device 1 that processes an image (captured image) taken by the camera 11, and a vehicle control device 2 that controls a vehicle based on the processing result of the image processing device 1.

The image processing device 1 and the vehicle control device 2 are computers (for example, a microcomputer), respectively, for example, an input unit, a central processing unit (CPU or MPU) which is a processor, and a read-only memory (ROM) which is a storage device. It also has a random access memory (RAM) and an output unit. Of these, the input unit converts various information input to the image processing device 1 and the vehicle control device 2 so that the CPU can calculate. The ROM is a recording medium in which a control program that executes arithmetic processing described later as appropriate and various information necessary for executing the arithmetic processing are stored, and the CPU is an input unit and a ROM according to the control program stored in the ROM. , Performs predetermined arithmetic processing on the signal taken from the RAM. From the output unit, commands for controlling the output target and information used by the output target are output. The storage device is not limited to the above-mentioned semiconductor memories such as ROM and RAM, and can be replaced with a magnetic storage device such as a hard disk drive, for example.

The image processing device 1 is a computer that executes external world recognition processing such as object detection based on an image (color image) captured by the camera 11, and includes an image acquisition unit 12, a brightness calculation formula changing unit 13, and brightness calculation. A unit 14 and a detection unit 15 are provided. The vehicle control device 2 is a computer that executes vehicle control processing based on the detection result of the detection unit 15 output by the image processing device 1, and comprises a light distribution control unit 16 and a constant speed traveling / inter-vehicle distance control unit 17. I have. Although the image processing device 1 and the vehicle control device 2 in FIG. 1 are composed of two different computers, they may be configured by an integrated computer.

The image acquisition unit 12 controls the camera 11, periodically sets the exposure time (shutter speed) and captures images, acquires a time series of color images, and executes a process of storing them in the image processing device 1. To do. A color image is defined by an RGB color model, and the color of each pixel constituting the color image is defined by a combination of brightness of red (R), green (G), and blue (B). In this embodiment, the brightness of each of R, G, and B is represented by an integer value from 0 to 255 (that is, 256 gradations), and the color of each pixel is represented by three values (R value, G value, B value). It is defined by the combination of. The color image is taken at a predetermined cycle, and for example, 30 frames may be taken per second.

The luminance calculation unit (luminance image generation unit) 14 performs a process of generating a luminance image (Y) from an image (color image) captured by the camera 11 by using the luminance calculation formula determined by the luminance calculation formula changing unit 13. Execute. The luminance image generated here is stored in the image processing device 1 and output to the detection unit 15 and the luminance calculation formula changing unit 13.

The luminance calculation formula changing unit 13 has a luminance index value of a predetermined region A located in the center of a luminance image generated in the past (for example, before a predetermined frame) by the luminance calculation unit 14, and a predetermined luminance index value to be detected by the detection unit 15. The brightness calculation formula used by the brightness calculation unit 14 according to the output of the detection unit 15 including the information of the object of the above and the output of the vehicle control device 2 including the information of the application executed by the vehicle control device 2. Executes the process of changing / determining. In this embodiment, four luminance calculation formulas (first to fourth luminance calculation formulas) described later are used, and the luminance calculation formula changing unit 13 is to select one from the four luminance calculation formulas. There is. Each luminance calculation formula is defined as the sum of the values obtained by multiplying each of the R value, the G value, and the B value by a predetermined coefficient, and the luminance calculation unit 14 uses this formula as the R value of the pixels constituting the color image. The brightness of the pixel is calculated by substituting the G value and the B value. A luminance image is obtained by calculating and assembling the luminance of each pixel of the image captured by the camera 11.

The predetermined area A is a predetermined area set for each luminance image. The predetermined area A can be set based on various rules, and among them, the predetermined area A based on the position of the vehicle traveling in front of the own vehicle may be particularly referred to as the front vehicle area A. FIG. 2 is an explanatory diagram of a predetermined area A. In the luminance image 71 shown in the upper row, the circumscribed rectangle 75 shows the vehicle in front (preceding vehicle) that was detected by the detection unit 15 until just before but could not be detected due to the overexposure of the image due to the tunnel exit. A predetermined area (front vehicle area) A1 is determined by defining a rectangle having a predetermined size based on the position of the vehicle in front (circumscribed rectangle 75). In the luminance image 72 shown in the middle row, the vehicle in front, which is also undetectable due to the blackout of the image due to the tunnel entrance, is shown by the circumscribing rectangle 76, and the position of the vehicle in front (external rectangle 76) is used as a reference. A predetermined area (front vehicle area) A1 is determined by defining a rectangle having a predetermined size. The luminance image 72 shown in the lower row shows the circumscribing rectangle 76 of the vehicle in front as in the luminance image 72 in the middle row, but the central portion of the luminance image (that is, the camera 11) is directly independent of the position of the vehicle in front. A predetermined area A2 is set as a closed area (for example, a rectangle) having a predetermined size at a predetermined position (the central portion of the captured image when the camera is directed in the traveling direction of the own vehicle). Although only the example of the preceding vehicle traveling in the same direction as the own vehicle has been described here as the front vehicle, the predetermined area A may be set based on the position of the oncoming vehicle traveling in the opposite direction to the own vehicle.

The brightness calculation formula changing unit 13 of the present embodiment can change the brightness calculation formula based on the brightness index value of the predetermined region A. Here, the luminance index value is an index value indicating the luminance as an aggregate of a plurality of pixels included in the predetermined region A, and is, for example, all the pixels included in the predetermined region A or a plurality of appropriately selected pixels. The average value of the brightness and the maximum or minimum brightness of all the pixels can be used. In the present embodiment, the average brightness of all the pixels included in the predetermined area A will be described as the brightness index value.

The detection unit 15 executes a process of detecting a predetermined object based on at least one of the color image acquired by the image acquisition unit 12 and the luminance image generated by the luminance calculation unit 14, and the detection result is controlled by the vehicle. Output to device 2. The "predetermined object" to be detected by the detection unit 15 may be determined by the type of application (program) executed by the vehicle control device 2. When the light distribution control application related to the light distribution control unit 16 is executed, the tail lights (red light emitters) of other vehicles (preceding vehicle, stopped vehicle, etc.) and the headlights of other vehicles (oncoming vehicles) are executed. (White light emitter) can be a predetermined object, and when these objects are detected, it can be considered that the vehicle has been detected. In addition, when the application of constant speed driving / inter-vehicle distance control (ACC: Adaptive Cruise Control) related to the constant-speed traveling / inter-vehicle distance control unit 17 is executed, the tail light of another vehicle (preceding vehicle, stopped vehicle, etc.) (Red illuminant) and tail lights (red illuminant) of other vehicles (preceding vehicle, stopped vehicle, etc.) taken under dark exposure conditions (for example, exposure time is relatively short) are the predetermined objects. Can be

The light distribution control unit 16 in the vehicle control device 2 executes light distribution control (light distribution control application) of the own vehicle headlights based on the detection result (vehicle detection result) of the detection unit 15 of the image processing device 1. As for the light distribution control, for example, when there is no preceding vehicle or oncoming vehicle in front of the own vehicle, the own vehicle headlight is set to high beam, and when there is a preceding vehicle or oncoming vehicle, the own vehicle headlight is set to low beam. There is something to do. In addition, when another vehicle is detected in front of the own vehicle, the light-shielding area of the own vehicle headlight is determined so that the other vehicle is excluded from the irradiation area of the own vehicle headlight, and the own vehicle is based on the light-shielding area. Some control the light distribution of the headlights. In the latter case, the vehicle is provided with an actuator (for example, multiple motors) that changes the optical axes of multiple headlights and a shade mechanism for blocking a part of the light of each headlight, and the light distribution is controlled. A control signal for them is output from the unit 16.

The constant-speed traveling / inter-vehicle distance control unit 17 keeps the traveling speed constant while maintaining a constant distance between the detected vehicle and the own vehicle based on the detection result (vehicle detection result) of the detection unit 15 of the image processing device 1. ACC (ACC application) that appropriately controls the mechanisms (steering control unit, engine control unit, brake control unit) corresponding to the steering control (steering control) and speed control (engine control, brake control) of the own vehicle so as to hold the vehicle. To execute.

-Light distribution control processing-
Next, the flow of the light distribution control process by the vehicle control system 100 will be described with reference to FIG. The vehicle control system 100 (image processing device 1 and vehicle control device 2) repeatedly executes the flowchart of FIG. 3 at a predetermined control cycle.

When the processing is started, the image acquisition unit 12 sets the exposure time of the image to be captured next to the camera 11 in step S1, and performs imaging at a predetermined timing to acquire a color image (captured image).

In step S2, the luminance calculation formula changing unit 13 executes the determination process of the luminance calculation formula.
The brightness calculation formula changing unit 13 of the present embodiment changes the brightness calculation formula according to the type of the detection object of the detection unit 15. FIG. 4 is a flowchart of the luminance calculation formula determination process (change process) by the luminance calculation formula changing unit 13 according to the present embodiment.

Here, the reference luminance calculation formula normally used by the luminance calculation unit 14 for generating a luminance image is referred to as a first luminance calculation formula. The first luminance calculation formula in this embodiment is represented by the following formula (1). In the equation (1), Y indicates the luminance value, and R, G, and B indicate the R value, G value, and B value of each pixel. That is, the first luminance calculation formula is defined as the sum of the values obtained by multiplying each of the R value, G value, and B value of any pixel in the captured image by a predetermined coefficient. The sum of the coefficients of R, G, and B is 1.

Y = 0.299R + 0.587G + 0.114B ... Equation (1)

When the process of FIG. 4 is started, the luminance calculation formula changing unit 13 inputs the type of the detection object of the detection unit 15 based on the detection result input from the detection unit 15 in step S21.

In step S22, the luminance calculation formula changing unit 13 determines whether or not the detection target in step S21 is the headlight (white light emitter) of the oncoming vehicle. Here, if the object to be detected is the headlight of an oncoming vehicle, the process proceeds to step S23, otherwise the process proceeds to step S24.

In step S23, the luminance calculation formula changing unit 13 sets the second luminance calculation formula as the luminance calculation formula. The second luminance calculation formula has a larger B value coefficient than the first luminance calculation formula, and a smaller R value and G value coefficient than the first luminance calculation formula. The second luminance calculation formula in this embodiment is represented by the following formula (2). Sensitivity to white light such as headlights differs between R, G, and B pixels, and there is a relationship of "G>R>B". Therefore, when the second luminance calculation formula is used to generate the luminance image, overexposure is reduced as compared with the case where the first luminance calculation formula is used.

Y = B ... Equation (2)

In step S24, the luminance calculation formula changing unit 13 determines whether or not the detection target in step S21 is the tail light (red light emitter) of the preceding vehicle. Here, if the object to be detected is the tail light of the preceding vehicle, the process proceeds to step S25, otherwise the process proceeds to step S26.

In step S25, the luminance calculation formula changing unit 13 sets the fourth luminance calculation formula as the luminance calculation formula. The fourth luminance calculation formula has a larger coefficient of R value than the first luminance calculation formula, and a smaller coefficient of G value and B value than the first luminance calculation formula. The fourth luminance calculation formula in this embodiment is represented by the following formula (4). Sensitivity to red light is improved by making the coefficient of the R value larger than that of the first luminance calculation formula. Therefore, when the fourth luminance calculation formula is used to generate the luminance image, the black crushing of the red light emitter is reduced as compared with the case where the first luminance calculation formula is used.

Y = R ... Equation (4)

In step S26, the luminance calculation formula changing unit 13 sets the first luminance calculation formula as the luminance calculation formula.

When the luminance calculation formula is determined in any of steps S23, S25, and S26 as described above, the luminance calculation formula changing unit 13 ends the process of FIG. 4 and starts the process of step S3 of FIG.

In step S3, the luminance calculation unit 14 is acquired by the image acquisition unit 12 in step S1 using the luminance calculation formula determined by the luminance calculation formula changing unit 13 in step S2 (that is, any of steps S23, S25, and S26). A luminance value Y is calculated for all the pixels in the color image, and a luminance image is generated from the calculated luminance value Y. The generated luminance image is output to the detection unit 15 and the luminance calculation formula changing unit 13.

In step S4, the detection unit 15 detects the vehicle candidate light spot from the color image acquired by the image acquisition unit 12. Specifically, first, the color conditions to be searched for in the color image are set according to the type of the detection object, and the color image is scanned to detect the pixels of the color corresponding to the conditions. Next, adjacent detection pixels are connected as the same group, and the area occupied by those pixel groups (groups) is regarded as a light spot. By this step, a plurality of light spots suitable for the type of the object to be detected can be detected.

By the way, the detection of the vehicle candidate light spot in step S4 has the following problems. That is, when the preceding vehicle tail light (see the inside of the rectangle of the image 51 in FIG. 5) is the detection target of the detection unit 15, the light spot is detected by searching for red in the color image. However, the same red color as the tail light may appear at the edge of the image of the originally orange reflector on the road (see the inside of the rectangle of image 52 in FIG. 5), and the tail is judged only by the degree of redness of the light spot. It is difficult to distinguish between a light and a reflector. Therefore, in the present embodiment, the determination based on the brightness of the subsequent step S5 is performed.

In step S5, the detection unit 15 determines whether or not each light spot detected in step S4 is a vehicle light (oncoming vehicle headlight or preceding vehicle taillight). Specifically, a representative pixel (for example, the brightest pixel) included in the area of the light spot is selected. Then, the luminance value of the pixel is confirmed by the luminance image generated in step S3, and it is determined whether or not the luminance value of the pixel is equal to or more than a predetermined threshold value determined for each detection object. If the luminance value of the pixel is equal to or greater than a predetermined threshold value, the process proceeds to step S6, and if the luminance value is less than the predetermined threshold value, the process proceeds to step S7.

In step S6, the detection unit 15 refers to a light spot whose brightness value is determined in step S5, and the light spot is a vehicle light (oncoming vehicle headlight or preceding vehicle taillight), and the vehicle related to the vehicle light. Is added to the effect that the light is to be shaded by the headlights of the own vehicle, and is output to the light distribution control unit 16.

In step S7, the detection unit 15 indicates that the light spot whose brightness value is determined in step S5 is not a vehicle light and the vehicle related to the vehicle light is not a light-shielding target of the own vehicle headlight. Information is added and output to the light distribution control unit 16.

In step S8, the light distribution control unit 16 determines a light-shielding area of the own vehicle headlight based on the light spot information obtained in step S6 or S7, and the light-shielding area is excluded from the irradiation area of the own vehicle headlight. The light distribution of the headlights of the own vehicle is controlled so as to be performed.

As described above, one cycle of the light distribution control process by the vehicle control system 100 is completed. The vehicle control system 100 waits for a predetermined control cycle, and then re-executes a series of processes from step S1.

<Action / effect>
Next, the operation / effect of the vehicle control system 100 configured as described above will be described.

First, the problem to be solved by the present invention will be described. In the determination based on the brightness value in step S5 described with reference to FIG. 3, a self-luminous light emitting body (for example, a preceding vehicle tail light or an oncoming vehicle headlight) and a reflected light type light emitting body (for example, a reflector) are discriminated. Can be difficult. FIG. 6 is a diagram showing the relationship between the distance and the brightness value of the pixels of the images 51 and 52 related to the tail light and the reflector under the same exposure conditions. As shown in this figure, the brightness values of the light spots of the tail light and the reflector both reach the maximum value at a short distance (150 m or less in the example in the figure) (the brightness value is saturated), and the brightness value becomes It cannot be distinguished by size. It can be improved by taking an image under an exposure condition that does not saturate the brightness value, but it is practically difficult to change the exposure condition because the image is shared with other functions of the vehicle control system.

Therefore, in the present embodiment, as shown in the flowchart shown in FIG. 4, the luminance calculation formula used when creating the luminance image is changed according to the type of the detection object of the detection unit 15.

Specifically, when the oncoming vehicle headlight is an object to be detected, the second luminance calculation formula (Y = B) is used in step S23. FIG. 7 is a diagram showing an example of the brightness image and the brightness distribution of the orange reflector on the road. The left side of the figure shows the case where the first brightness calculation formula is used, and the right side of the figure shows the second brightness calculation. The case where the formula is used is shown. As shown on the left side of FIG. 7, when the first luminance calculation formula is used, one of the peaks of the luminance value distribution exists at the maximum value (255), and in the processing of S5 based on the luminance, the oncoming vehicle head It turns out that it is difficult to distinguish from the light. As described above, B among R, G, and B has the lowest sensitivity to white light. When the brightness value is calculated using the second brightness calculation formula in which the B value coefficient is larger than the first brightness calculation formula and the R value and G value coefficients are smaller, the first brightness calculation is performed as shown on the right side of FIG. The brightness values of the light spots are distributed at lower values than when the equation is used, and in the example of FIG. 7, the distribution of the brightness values can be distributed to values less than 255 (values on the left side in the figure). This makes it easier to distinguish between oncoming vehicle headlights and reflectors based on the brightness value, and it is not possible to obtain an image with exposure conditions suitable for detecting oncoming vehicle headlights, which are white self-luminous light emitters. However, the detection rate of the light emitting body can be improved without increasing the cost.

Similarly, when the tail light of the preceding vehicle is the object to be detected, the fourth luminance calculation formula (Y = R) is used in step S25. When the fourth brightness calculation formula, which has a larger R value coefficient than the first brightness calculation formula and a smaller G value and B value coefficient, is used, the pixels having a high R value related to the tail light are emphasized as high brightness. At the same time, the brightness value of the light spot is distributed at a lower value than when the first brightness calculation formula is used. This makes it easy to distinguish between the tail light of the preceding vehicle and the tail light of the reflector based on the brightness value, and the detection rate of the tail light of the preceding vehicle, which is a red self-luminous light emitter, can be improved.

-Constant speed driving / inter-vehicle distance control (ACC)-
The change of the brightness calculation formula by the brightness calculation formula change unit 13 described above is, for example, when the captured image is blacked out or overexposed at the entrance or exit of the tunnel during execution of ACC and the preceding vehicle in the distance is lost. It can also be used (see images 71 and 72 in FIG. 2). FIG. 8 is an example of a flowchart of the luminance calculation formula determination process (change process) by the luminance calculation formula changing unit 13 in this case. The luminance calculation formula changing unit 13 repeatedly executes this flowchart at a predetermined control cycle.

When the process of FIG. 8 is started, the luminance calculation formula changing unit 13 changes from the state in which another vehicle is detected to the undetectable state based on the detection result input from the detection unit 15 in step S31. Judge whether or not. Then, if it changes undetectable, the process proceeds to step S32, and if it does not change (that is, when the detection of another vehicle continues), the process proceeds to step S40.

In step S32, the luminance calculation formula changing unit 13 reads as much as possible the latest luminance image generated based on the color image taken by the camera 11 in a state where another vehicle is detected. A color image may be read instead of the luminance image.

In step S33, the luminance calculation formula changing unit 13 sets a predetermined region (front vehicle region) A with reference to the position of another vehicle in the luminance image read in step S32. As an example of the setting method of the predetermined area A, there is the one described with reference to FIG. When the color image is read in step S32, the predetermined area A may be set based on the position of another vehicle on the image.

In step S34, the luminance calculation formula changing unit 13 calculates the luminance index value of the predetermined region A in the luminance image read in step S32. When the color image is read in step S32, the luminance index value of the predetermined region A on the luminance image corresponding to the color image is calculated.

In step S35, the luminance calculation formula changing unit 13 determines whether or not the luminance index value of the predetermined region A is equal to or less than the first threshold value (Y1) and equal to or greater than the second threshold value (Y2). The first threshold value (Y1) is the threshold value of the luminance defined to consider that overexposure has occurred in the color image (captured image), and the second threshold value (Y2) is similarly considered to have caused blackout. It is a threshold value of brightness defined for this.
However, the first threshold value (Y1) is set to a value larger than the second threshold value (Y2) (that is, Y1> Y2). If the luminance index value of the predetermined region A is equal to or greater than the second threshold value and equal to or less than the first threshold value, it can be considered that the luminance is appropriate. If the luminance index value of the predetermined region A is equal to or less than the first threshold value (Y1) and equal to or greater than the second threshold value (Y2), the process proceeds to step S36, and if not, the process proceeds to step S37.

In step S36, the luminance calculation formula changing unit 13 sets the first luminance calculation formula as the luminance calculation formula.

In step S37, the luminance calculation formula changing unit 13 determines whether or not the luminance index value of the predetermined region A is larger than the first threshold value (Y1). If the luminance index value of the predetermined region A is larger than the first threshold value (Y1), the process proceeds to step S38, and if not (that is, when the luminance index value of the predetermined region A is smaller than the second threshold value (Y2)), the process proceeds to step S38. The process proceeds to step S39.

In step S38, the luminance calculation formula changing unit 13 sets the second luminance calculation formula as the luminance calculation formula.

In step S39, the luminance calculation formula changing unit 13 sets the third luminance calculation formula as the luminance calculation formula. The third luminance calculation formula has a larger G value coefficient than the first luminance calculation formula, and a smaller R value and B value coefficient than the first luminance calculation formula. The third luminance calculation formula in this embodiment is represented by the following formula (3). As described above, the sensitivity to white light differs between R, G, and B pixels, and the relationship is "G>R>B". Therefore, when the third luminance calculation formula is used to generate the luminance image, the blackout is reduced as compared with the case where the first luminance calculation formula is used.
Y = G ... Equation (3)
In step S40, the luminance calculation formula changing unit 13 sets the first luminance calculation formula as the luminance calculation formula.

<Action / effect>
The action / effect when a luminance image is generated by using the luminance calculation formula determined by the luminance calculation formula changing unit 13 configured as described above will be described.

First, when the color image (captured image) is overexposed at the exit of the tunnel and the other vehicle in the distance is lost, the luminance calculation formula changing unit 13 sets the second luminance calculation formula to the luminance calculation formula in step S38. The luminance calculation unit 14 uses this second luminance calculation formula to generate a luminance image. In this luminance image, overexposure is reduced by the second luminance calculation formula, so that the possibility of detecting the lost vehicle in the color image is improved. As a result, the detection rate of other vehicles can be improved without increasing the cost even in a situation where an image captured under exposure conditions suitable for detecting other vehicles when overexposure occurs cannot be obtained.

Further, when the color image (captured image) is blacked out at the entrance of the tunnel and the other vehicle in the distance is lost, the luminance calculation formula changing unit 13 sets the third luminance calculation formula to the luminance calculation formula in step S39. The luminance calculation unit 14 uses this third luminance calculation formula to generate a luminance image. In this luminance image, the blackout is reduced by the third luminance calculation formula, so that the possibility of detecting the lost vehicle in the color image is improved. As a result, the detection rate of other vehicles can be improved without increasing the cost even in a situation where an image captured under exposure conditions suitable for detecting another vehicle when blackout occurs cannot be obtained.

If overexposure or underexposure does not occur, or if other vehicles are not lost, the same first luminance calculation formula as usual is set, so that no inconvenience occurs due to the change in the luminance calculation formula.

-Modification example 1-
By the way, in the example of FIG. 8, the predetermined area A is set based on the position of the lost vehicle, but it is predetermined at a predetermined position in the central portion of the luminance image regardless of the position of the vehicle. A rectangle of the same size may be set as a predetermined area to determine the brightness calculation formula. FIG. 9 shows an example of a flowchart of the luminance calculation formula determination process (change process) by the luminance calculation formula changing unit 13 in this case. The luminance calculation formula changing unit 13 repeatedly executes this flowchart at a predetermined control cycle. The same processing as in FIG. 8 is designated by the same reference numerals, and the description thereof will be omitted.

When the process of FIG. 9 is started, the luminance calculation formula changing unit 13 is in step S41. In step S41, the luminance calculation formula changing unit 13 is the latest possible luminance image generated based on the color image captured by the camera 11. Read things. A color image may be read instead of the luminance image.

In step S42, the luminance calculation formula changing unit 13 calculates the luminance index value of the predetermined predetermined region A in the luminance image read in step S41. When the color image is read in step S41, the luminance index value of the predetermined region A on the luminance image corresponding to the color image is calculated.

Since the processing after step 35 is the same as that in FIG. 8, the description thereof will be omitted.

Even if the brightness calculation formula changing unit 13 is configured in this way, even in a situation where it is not possible to obtain an image of exposure conditions suitable for detecting another vehicle when overexposure or underexposure occurs, the cost of the other vehicle is not increased. The detection rate can be improved.

-Modification example 2-
The flowcharts of FIGS. 8 and 9 may be combined with the flowchart of FIG. FIG. 10 shows an example of a flowchart of the luminance calculation formula determination process (change process) by the luminance calculation formula changing unit 13 in this case. In FIG. 10, when NO is determined in step S31, the process proceeds to step S21 in FIG. Even if the brightness calculation formula changing unit 13 is configured in this way, even in a situation where an image captured with exposure conditions suitable for detecting a predetermined detection object cannot be obtained, the detection of the predetermined detection object is performed without increasing the cost. The rate can be improved. Although FIG. 10 shows an example in which the flowcharts of FIGS. 8 and 4 are combined, it goes without saying that the flowcharts of FIGS. 9 and 4 may be combined.

Finally, the four luminance calculation formulas used in each embodiment are summarized. FIG. 11 is a table summarizing the contents of each luminance calculation formula and the conditions under which each luminance calculation formula is selected.

As shown in this figure, as a condition for selecting the second luminance calculation formula (Y = B) used for reducing overexposure, for example, the luminance index value of the predetermined region (front vehicle region) A is the first. When it is higher than one threshold value (Y1), when the preceding vehicle becomes undetectable due to overexposure of the captured image (for example, at the tunnel exit), when the detection target of the detection unit 15 is the headlight of the oncoming vehicle (that is, When the headlight and the reflector are distinguished).

Further, as a condition for selecting the third luminance calculation formula (Y = G) used for reducing blackout, for example, the luminance index value of the predetermined region (front vehicle region) A is the second threshold value (Y2). If it is lower, the preceding vehicle may become undetectable due to blackout of the captured image (for example, under a tunnel entrance or overpass).

Further, as a condition for selecting the fourth luminance calculation formula (Y = R) used for detecting a dark red light, for example, the detection target of the detection unit 15 is far away (for example, a distance of 500 m or more from the own vehicle). ), The detection target of the detection unit 15 is the tail light of a traveling vehicle at a short distance (for example, within 50 m from the own vehicle) (when the exposure condition is darker than in the former case). There is.

The coefficient in the first luminance calculation formula is only an example. For example, the second decimal place may be rounded off to obtain "Y = 0.3R + 0.6 + 0.1B".

− Other −
The present invention is not limited to the above-described embodiment, and includes various modifications within a range that does not deviate from the gist thereof. For example, the present invention is not limited to the one including all the configurations described in the above-described embodiment, and includes the one in which a part of the configurations is deleted. Further, it is possible to add or replace a part of the configuration according to one embodiment with the configuration according to another embodiment.

Further, for each configuration related to the processing device 1 and the control device 2 and the functions and execution processing of each configuration, a part or all of them are designed by hardware (for example, the logic for executing each function is designed by an integrated circuit). Etc.) may be realized. Further, the configuration related to the above devices 1 and 2 may be a program (software) in which each function related to the configuration of the device is realized by reading and executing by an arithmetic processing unit (for example, a CPU). Information related to the program can be stored in, for example, a semiconductor memory (flash memory, SSD, etc.), a magnetic storage device (hard disk drive, etc.), a recording medium (magnetic disk, optical disk, etc.), or the like.

Further, in the description of each of the above embodiments, the control lines and information lines are understood to be necessary for the description of the embodiment, but not all control lines and information lines related to the product are necessarily used. Does not always indicate. In reality, it can be considered that almost all configurations are interconnected.

1 ... Image processing device, 11 ... Camera, 12 ... Image acquisition unit, 13 ... Luminance calculation formula change unit, 14 ... Luminance calculation unit, 15 ... Vehicle detection unit, 16 ... Light distribution control unit, 17 ... Constant speed running / inter-vehicle distance Distance control unit, 100 ... Vehicle control system

Claims (11)

Imaging device and
Image processing including a luminance calculation unit that generates a luminance image from an image captured by the image pickup apparatus using a luminance calculation formula, and an image processing device having a detection unit that detects a predetermined object based on the luminance image. It's a system
The image processing device changes the luminance calculation formula used by the luminance calculation unit according to the luminance index value of the predetermined region in the luminance image or the type of the predetermined object detected by the detection unit. An image processing system characterized by further including a calculation formula changing unit. In the image processing system of claim 1,
When the luminance index value of the predetermined region in the luminance image exceeds the first threshold value, the luminance calculation formula changing unit has the luminance index value of the predetermined region in the luminance image equal to or greater than the second threshold value smaller than the first threshold value. Changing the luminance calculation formula to a second luminance calculation formula that reduces overexposure of the luminance image as compared with the first luminance calculation formula used by the luminance calculation unit as the luminance calculation formula when the first threshold value or less is reached. An image processing system characterized by. In the image processing system of claim 1,
The luminance calculation formula changing unit changes from a state in which the predetermined object is detected by the detection unit to a state in which it cannot be detected, and the luminance index value of the predetermined region in the luminance image sets the first threshold value. When it exceeds, the reference luminance calculation formula used by the luminance calculation unit as the luminance calculation formula when the luminance index value of the predetermined region in the luminance image is equal to or greater than the second threshold smaller than the first threshold and equal to or less than the first threshold. An image processing system characterized by changing the luminance calculation formula to a second luminance calculation formula that reduces overexposure of the luminance image. In the image processing system of claim 1,
When the predetermined object of the detection unit is a white light emitter, the luminance calculation formula changing unit has a luminance image higher than the first luminance calculation formula normally used by the luminance calculation unit as the luminance calculation formula. An image processing system characterized in that the brightness calculation formula is changed to a second brightness calculation formula that reduces overexposure. In the image processing system of claim 2,
The captured image is defined by an RGB color model.
The first luminance calculation formula is defined as the sum of the values obtained by multiplying each of the R value, G value, and B value of any pixel in the captured image by a predetermined coefficient.
The second luminance calculation formula is an image processing system characterized in that the B value coefficient is larger than that of the first luminance calculation formula and the R value and G value coefficients are smaller than those of the first luminance calculation formula. In the image processing system of claim 1,
When the brightness index value of the predetermined region in the luminance image is less than the second threshold value, the luminance calculation formula changing unit has the luminance index value of the predetermined region in the luminance image equal to or greater than the second threshold value and more than the second threshold value. Changing the brightness calculation formula to a third brightness calculation formula that reduces blackout of the brightness image more than the first brightness calculation formula used by the brightness calculation unit as the brightness calculation formula when the first threshold value is large or less. An image processing system characterized by. In the image processing system of claim 1,
The luminance calculation formula changing unit changes from a state in which the predetermined object is detected by the detection unit to an undetectable state, and the luminance index value of the predetermined region in the luminance image is less than the second threshold value. When the brightness index value of the predetermined region in the brightness image is equal to or higher than the second threshold value and equal to or lower than the first threshold value larger than the second threshold value, the first brightness calculation used by the brightness calculation unit as the brightness calculation formula is used. An image processing system characterized in that the luminance calculation formula is changed to a third luminance calculation formula that reduces blackout of the luminance image rather than the formula. In the image processing system of claim 6,
The captured image is defined by an RGB color model.
The first luminance calculation formula is defined as the sum of the values obtained by multiplying each of the R value, G value, and B value of any pixel in the captured image by a predetermined coefficient.
The third luminance calculation formula is an image processing system characterized in that the coefficient of G value is larger than that of the first luminance calculation formula and the coefficients of R value and B value are smaller than those of the first luminance calculation formula. In the image processing system of claim 1,
When the predetermined object of the detection unit is a red light emitter, the luminance calculation formula changing unit has a luminance image higher than the first luminance calculation formula normally used by the luminance calculation unit as the luminance calculation formula. An image processing system characterized in that the luminance calculation formula is changed to a fourth luminance calculation formula for reducing blackout. In the image processing system of claim 9,
The captured image is defined by an RGB color model.
The first luminance calculation formula is defined as the sum of the values obtained by multiplying each of the R value, G value, and B value of any pixel in the captured image by a predetermined coefficient.
The fourth luminance calculation formula is an image processing system characterized in that the R value coefficient is larger than that of the first luminance calculation formula and the G value and B value coefficients are smaller than those of the first luminance calculation formula. Imaging device and
An image processing device having a luminance calculation unit that generates a luminance image from an image captured by the image pickup apparatus using a luminance calculation formula, and a detection unit that detects a predetermined object based on the luminance image.
It is a light distribution control system including a light distribution control unit that performs light distribution control based on a light-shielding area of the own vehicle headlight determined based on the detection result of the detection unit.
The image processing device changes the luminance calculation formula used by the luminance calculation unit according to the luminance index value of the predetermined region in the luminance image or the type of the predetermined object detected by the detection unit. A light distribution control system characterized by further including a calculation formula changing unit.

Images