JP2016009936A - Vehicle surroundings monitoring apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a vehicle surroundings monitoring apparatus for preventing a flashing display of a lighting, such as a headlight, to be continuously illuminated, while displaying a lighting, such as a turn-signal, to be flashed, in displaying a pickup image of the surroundings of a vehicle on display means disposed inside a cabin.SOLUTION: A monitoring apparatus includes: imaging-means for picking up the surroundings of a vehicle; and display means, disposed in a cabin of the vehicle, for displaying the image picked up by the imaging-means. In a case where a brightness change amount of given color data among images in which levels of brightness of a corresponding region have variation equaling or exceeding a given level, among images that have been continuously picked up, or in a case where an interval in which levels of brightness of a corresponding region have variation equaling or exceeding a given level is shorter than a given interval, the display means corrects the brightness of the corresponding region of an image in which the brightness of the corresponding region is equal to or less than a given brightness value, among the images that have been continuously picked up, and displays the corrected image.

Description

  The present invention relates to a vehicle periphery monitoring device that displays an image of the periphery of a vehicle captured by an in-vehicle imaging device on a display device in a vehicle interior.

  2. Description of the Related Art Conventionally, there is known a vehicular visual assistance device that displays an image captured by a camera mounted on the outside of a vehicle compartment on a display device (such as a head-up display) in the vehicle interior (for example, Patent Document 1). Thereby, it is possible to make it possible to visually recognize a place (a blind spot) that cannot be viewed with a side mirror or the like, or to eliminate the side mirror itself and improve the aerodynamic performance of the vehicle.

JP 2008-141574 A

  By the way, at night, illumination such as headlights of other vehicles around the vehicle may be captured and displayed on a display device by an imaging unit such as a camera provided outside the passenger compartment. In some cases, lighting such as headlights is duty controlled in order to adjust the illuminance. In this case, lighting such as headlights blinks in a very short period that cannot be visually recognized by humans (repetition of turning on and off). ) Is dimmed, and it is visually recognized as a constantly lit illumination by humans.

  However, since the imaging cycle of the imaging means such as a camera is longer than the blinking cycle of the headlight, when the moment when the headlight is turned off is imaged and displayed on the display device, the headlight turns off even for humans. There is a possibility that the headlight can be seen and the headlight appears to blink.

  Here, FIG. 1 is a schematic diagram showing the relationship between the blinking cycle of the headlight and the display display cycle. The upper diagram shows the flashing mode of the headlight by the headlight ON / OFF signal, and the lower diagram shows the state of the headlight (lit or extinguished) displayed on the display corresponding to the time-series change. A display as a display device displays an image of 60 frames per second (frequency: 60 Hz), and the headlight blinks at a shorter cycle than the display. Further, since the imaging cycle of the camera is twice the display cycle of the display (frequency 30 Hz), the display displays the images captured by the camera twice. Referring to FIG. 1, at times t0 and t1, the camera takes an image of the lit headlight, and the lit headlight is displayed on the display for four frames from time t0. However, at time t2, the camera images the headlight that has been turned off, and the headlight that has been turned off for two frames from time t2 is also displayed on the display. In this way, due to the discrepancy between the blinking cycle of the headlight and the imaging cycle of the camera, the actual headlight was displayed on the display even though it was blinking at a very short cycle that is not visible to humans. There is a risk that the headlight may appear to blink at a period that is visible to humans.

  Although it is conceivable to detect the headlight position of another vehicle and suppress blinking by image correction, there is a possibility that the blinking of a turn signal lamp or the like originally intended to blink will be corrected together. .

  Accordingly, in view of the above problems, when displaying an image around the vehicle imaged by the imaging means mounted on the outside of the passenger compartment on the display means provided in the passenger compartment, lighting such as a turn signal for blinking is blinked. It is an object of the present invention to provide a vehicle periphery monitoring device capable of preventing blinking display of lighting such as a headlight that is always lit while displaying.

In order to achieve the above object, in one embodiment, a vehicle periphery monitoring device includes:
A vehicle periphery monitoring device comprising: an imaging unit that images a vehicle periphery; and a display unit that is disposed inside the vehicle and displays an image captured by the imaging unit,
The display means includes
The brightness of corresponding parts between images continuously picked up by the imaging means is periodically changing more than a predetermined value, and the corresponding parts of the continuously picked-up images When the luminance change amount of the predetermined color data between the images in which the luminance changes more than the predetermined value is equal to or larger than the predetermined variation amount, or the cycle in which the luminance change of the corresponding part occurs is predetermined. When the period is shorter than the period, the luminance of the corresponding part in the image in which the luminance of the corresponding part is equal to or lower than a predetermined luminance value among the continuously captured images is corrected, and the corrected image is displayed. It is characterized by that.

  According to the present embodiment, when displaying an image around the vehicle imaged by the imaging means mounted on the outside of the vehicle compartment on the display means provided in the vehicle interior, illumination such as a turn signal for blinking is provided. It is possible to provide a vehicle periphery monitoring device capable of preventing blinking display of lighting such as a headlight intended to be constantly lit while displaying blinking.

It is a figure explaining an example displayed so that the headlight of the vehicle imaged with the camera may blink on a display. It is a block diagram which shows an example of a structure of a vehicle periphery monitoring apparatus. It is the schematic which shows the example of arrangement | positioning of the camera mounted in the vehicle. It is a figure explaining the technique which suppresses blinking of the headlight by a vehicle periphery monitoring apparatus. It is a flowchart which shows an example of the operation | movement (detection of the luminance change between the front and back frames) by a vehicle periphery monitoring apparatus (image analysis part). It is a flowchart which shows an example of the operation | movement (extraction of a correction | amendment object site | part) by a vehicle periphery monitoring apparatus (image analysis part). It is a flowchart which shows an example of the operation | movement (image correction) by a vehicle periphery monitoring apparatus (image processing part).

  Hereinafter, embodiments for carrying out the invention will be described with reference to the drawings.

  FIG. 2 is a block diagram illustrating an example of the configuration of the vehicle periphery monitoring device 1. The vehicle periphery monitoring device 1 is mounted on a vehicle, images the periphery of the vehicle, and displays the captured image on a display unit provided in the vehicle interior.

  The vehicle periphery monitoring device 1 includes a camera (imaging means) 10 and a display (display means) 20.

  The camera 10 is an imaging unit that is mounted on the outside of the passenger compartment and images the periphery of the vehicle. FIG. 3 is a schematic diagram illustrating an arrangement example of cameras mounted on the vehicle. As in this example, the camera 10 captures the rear from the side of the vehicle instead of the outer mirror (side mirror). May be arranged. Then, by displaying the image of the camera 10 on the display 20 in the vehicle interior, the driver can drive while confirming the lateral side of the vehicle. The camera 10 continuously captures images around the vehicle (for example, 30 images per second) and outputs image information (image signal) to the display 20 every predetermined imaging cycle.

  The display 20 is a display means for displaying an image around the vehicle imaged by the camera 10, and includes an image capture unit 21, a VRAM 22, an image analysis unit 23, a DRAM 24, an image processing unit 25, an image display unit 26, and the like. .

  The image capturing unit 21 is a unit that captures an image signal received from the camera 10 in a format that can be used in the display 20. Specifically, the image signal received from the camera 10 is converted into a video processing format by the image processing unit 25 in the display 20 and stored in the VRAM 22.

  The VRAM 22 is a storage unit for temporarily storing an image (information) captured by the image capturing unit 21 and subjected to format conversion.

  The image analysis unit 23 performs image analysis on the image temporarily stored in the VRAM 22. Specifically, a part (correction target part) that is a target of image correction on an image captured by the camera 10 is extracted. The image analysis unit 23 stores the image analysis result (extracted correction target portion or the like) in the DRAM 24. Details of the image analysis processing will be described later.

  The DRAM 24 is a storage unit that stores an image analysis result (such as a correction target part) by the image analysis unit 23.

  The image processing unit 25 performs image processing (image correction) on a necessary part based on the correction target part extracted by the image analysis part 23. An image signal corresponding to an image after image processing (including a case where image correction is not performed) is output to the image display unit 26. Details of the image correction processing will be described later.

  The image display unit 26 displays an image corresponding to the image signal received from the image processing unit 25.

  Next, an image analysis (extraction of a correction target part) and an image processing (image correction) operation by the vehicle periphery monitoring apparatus 1 (image analysis unit 23, image processing unit 25) according to the present embodiment will be specifically described.

  FIG. 4 is a diagram for explaining an overview of image analysis (extraction of a correction target portion) and image processing (image correction) operations performed by the vehicle periphery monitoring apparatus 1 (image analysis unit 23, image processing unit 25). 4 (a) and 4 (b) both show a state (for five frames) in which an image obtained by continuously capturing another vehicle (headlight lighting state) from the front by the camera 10 is displayed on the display 20. FIG. FIG. 4A shows a state before image correction, and FIG. 4B shows a state after image correction. Note that the headlights of other vehicles are repeatedly turned on and off at a cycle that is not visible to humans by duty control.

  Referring to FIG. 4A, in the third and fourth frames out of the first to fifth frames, the headlights in an extinguished state captured by the camera 10 are displayed. As described above, if the image captured by the camera 10 is displayed on the display 20 as it is, the headlight may appear to blink.

  Therefore, the image analysis unit 23 extracts a part where the luminance greatly changes periodically, that is, a part to be subjected to image correction (correction target part). Specifically, the luminance change amount of the corresponding part is calculated between consecutively captured frames (images), and the part in which the luminance significantly changes periodically (dudy-controlled head in each image) Light part, etc.) are extracted. In addition, the site | part corresponding between images means the site | part corresponding to the same part of the imaged object (imaged body) contained in the image imaged continuously. For example, when the vehicle and the imaged object are both stationary, they become parts of the same coordinates in each image. Further, when at least one of the vehicle and the object to be imaged is moving, the coordinates on the image of the same object to be imaged change between the images, so that the same part of the same object to be imaged in each image Become. In addition, when at least one of the vehicle and the object to be imaged is moving, it is preferable to specify a corresponding part between continuously captured images by using a motion vector or the like.

  In addition, in order to prevent a part such as a turn signal intended for blinking from being extracted and set as a correction target part, the luminance of predetermined color data is a periodicity among corresponding parts between consecutively captured images. It is good to extract the part which has changed greatly. For example, since the turn signal is often an orange color or a color close to yellow, the B value (B component) is utilized by utilizing the point that the luminance (B value) of the B component is relatively small in the RGB data. A portion where the amount of change in luminance) is greater than or equal to a predetermined value and the luminance greatly changes periodically may be extracted. Also, the turn signal blinking cycle is sufficiently longer than the duty control blinking cycle of headlights, etc., so the part where the brightness changes greatly in a cycle sufficiently shorter than the turn signal blinking cycle is extracted. May be.

  Subsequently, the image processing unit 25 performs image correction (luminance correction) on the correction target portion extracted by the image analysis unit 23, if necessary. In the first frame, the second frame, and the fifth frame in FIG. 4A, the state in which the headlight is turned on is displayed, and no correction is necessary. On the other hand, as shown in FIG. 4B, the third and fourth frame correction target parts (headlight parts) in which the headlights are turned off are raised to a luminance equivalent to that when the headlights are turned on. Make corrections. For example, using the luminance average value of the correction target part whose luminance is equal to or higher than a threshold value (in a state where the headlights are lit) among the correction target parts for a predetermined frame retroactively from the latest frame to be corrected. Brightness correction may be performed.

  Here, a specific example of image analysis and image processing operations by the vehicle periphery monitoring apparatus 1 (image analysis unit 23, image processing unit 25) shown in FIG. 4 will be described. In the following example, the description will be made on the assumption that the vehicle including the vehicle periphery monitoring device 1 and the object (object to be imaged) captured by the camera 10 are both stationary.

  5-7 is a figure which shows the specific example (example of processing flow) of operation | movement by the vehicle periphery monitoring apparatus 1 (the image analysis part 23, the image process part 25). FIG. 5 is a flowchart showing an example of an operation (detection of a luminance change between the preceding and following frames) by the vehicle periphery monitoring device 1 (image analysis unit 23). FIG. 6 is a flowchart illustrating an example of an operation (extraction of a correction target part) by the vehicle periphery monitoring apparatus 1 (image analysis unit 23). FIG. 7 is a flowchart illustrating an example of an operation (image correction) performed by the vehicle periphery monitoring device 1 (image processing unit 25). 5 to 7, an image (image signal) captured by the camera 10 is displayed on the display 20 (image) in a state where the camera 10 can capture an image and display the captured image on the display 20. It may be executed each time the capture unit 21) receives. The processing flow shown in FIGS. 5 to 7 is executed for each coordinate of an image captured by the camera 10 (image displayed on the display 20). In the following description, an image that is the target of each processing flow is executed. The coordinates are called target coordinates.

  Referring to FIG. 5, in step S101, color data (RGB data) between the previous and next frames of the latest image (frame) continuously captured by the camera 10 and the immediately preceding image (frame) for the target coordinates. Compare.

  In step S102, it is determined whether or not the luminance change amount dY between the previous and next frames is greater than or equal to the threshold value dYTH for the target coordinates. If the luminance change amount dY of the target coordinates is equal to or greater than the threshold value dYTH, the process proceeds to step S103. The threshold value dYTH may be determined based on, for example, the difference between the brightness when the headlight is turned off and the brightness when the headlight is turned on.

  In step S103, it is determined whether or not the change amount dB of the B value between the preceding and following frames is greater than or equal to the threshold dBTH in the RGB data for the target coordinates. If the change amount dB of the B value of the target coordinate is equal to or greater than the threshold dBTH, the process proceeds to step S104. The threshold dBTH may be determined based on, for example, the difference between the B values when the turn signal is turned on and off.

  In step S104, a value different from the value recorded at the address corresponding to the target coordinate for the immediately preceding frame among the values a and b is recorded at the address corresponding to the target coordinate on the DRAM 24, and this time for the target coordinate is recorded. The process ends. For example, if the value a is recorded at the address corresponding to the target coordinate for the immediately preceding frame, the value b is recorded at the address corresponding to the target coordinate for the current frame (latest frame).

  On the other hand, if the luminance change amount dY of the target coordinate is smaller than the threshold value dYTH in step S102, or if the B value variation amount dB of the target coordinate is smaller than the threshold value dBTH in step S103, the process proceeds to step S105.

  In step S105, the same value as the value recorded at the address corresponding to the target coordinate for the previous frame is recorded among the values a and b at the address corresponding to the target coordinate on the DRAM 24, and this time for the target coordinate is recorded. The process ends. For example, if the value a is recorded at the address corresponding to the target coordinate for the immediately preceding frame, the value a is recorded at the address corresponding to the target coordinate for the current frame (latest frame).

  In this way, when the luminance between the frames before and after the target coordinate changes greatly, the values a and b are recorded historically while switching, thereby periodically in the processing flow shown in FIG. It can be determined whether or not the luminance of the target coordinates has changed significantly. That is, a part (coordinate) that repeatedly turns on and off, such as a headlight, can be extracted as a correction target. Even when the luminance between the previous and next frames of the target coordinate changes greatly, if the change amount dB of the B value between the previous and next frames is small, the turn signal is recorded by switching the values a and b without switching. Thus, it is possible to prevent extraction of blinking such as. That is, as described above, the turn signal is often recorded with a relatively small B value, so that when the B value change amount dB is smaller than the threshold dBTH, the values a and b are recorded without switching. In the processing flow shown in FIG. 6 to be described later, it is not determined that the luminance of the target coordinates is periodically changing greatly, and a part such as a turn signal can be excluded from the correction target.

  Next, referring to FIG. 6, in step S201, the number of consecutive times of the value a recorded at the address corresponding to the target coordinate in the processing flow shown in FIG. Count Na (i). Note that i means an integer of 1 or more, and is a label value when there are a plurality of locations where the value a continues for the predetermined frame. For example, when there are two locations where the value a continues, Na (1) and Na (2) are counted.

  In step S202, it is determined whether or not the consecutive number Na (i) of the value a is smaller than the threshold value NTH. When all the consecutive times Na (i) of the value a are smaller than the threshold value NTH, the process proceeds to step S203.

  In step S203, as in step S201, the number of consecutive times Nb (i) of the value b recorded at the address corresponding to the target coordinate is counted for a predetermined number of consecutive frames including the latest frame. Note that i is an integer equal to or greater than 1 as in the case of the number of consecutive times Na (i) of the value a, and is a label value when there are a plurality of locations where the value b continues for the predetermined frame.

  In step S204, it is determined whether the number of consecutive times Nb (i) of the value b is smaller than the threshold value NTH. If all the consecutive times Nb (i) of the value b are smaller than the threshold value NTH, the process proceeds to step S205.

  In step S205, it is determined whether or not a state in which the number of consecutive times Na (i) of value a is equal to the number of consecutive times Nb (i) of value b continues for a predetermined number of times (e times), that is, periodically It is determined whether or not there is a large luminance change in the coordinates. Specifically, it is preferable to determine whether or not a state in which Na (i) ± d and Nb (i) ± d are equal and repeated is continued e times or more by using a parameter value d considering an error or the like. . If the determination condition is satisfied, the process proceeds to step S206.

  In step S206, assuming that the target coordinate is a part that periodically turns on and off, such as a headlight, the target coordinate is set as a correction target part, and the current processing for the target coordinate is terminated.

  On the other hand, if the determination condition is not satisfied in step S205, the process proceeds to step S207. Similarly, if the number of consecutive times Na (i) of the value a is greater than or equal to the threshold value NTH in step S202, or the number of consecutive times Nb (i) of the value b is greater than or equal to the threshold value NTH in step S204, The process proceeds to S207.

  In step S207, assuming that the target coordinates are not parts that are periodically turned on and off, such as a headlight, the target coordinates are not set as correction target parts, and the current process is terminated.

  As described above, the target coordinates are periodically changed depending on whether or not the consecutive times Na (i) and Nb (i) of the values a and b recorded historically in the processing flow of FIG. It can be determined whether or not the luminance of the target coordinates has changed significantly. And when the brightness | luminance of an object coordinate changes large periodically, an object coordinate can be specified as a correction | amendment object site | part. Further, when the number of consecutive times Na (i) and Nb (i) of the values a and b is equal to or greater than the threshold value NTH, the target coordinate is corrected regardless of whether or not the luminance of the target coordinate periodically changes. By not, turn signals having a long blinking cycle can be excluded from the correction target part.

  In order to exclude the turn signal and the like from the correction target, step S103 is provided in the processing flow of FIG. 5, and steps S202 and S204 are provided in the processing flow of FIG. Etc. may be excluded from the correction target.

  Subsequently, referring to FIG. 7, in step S301, the luminance average value AveY when the luminance value Y is greater than or equal to the threshold value YTH is calculated for the target coordinates, among the consecutive predetermined x frames including the latest frame, The average value AveY is recorded in the DRAM 24.

  In step S302, it is determined whether the target coordinates are a correction target part. In the processing flow of FIG. 6, when the target coordinate is a correction target part, the process proceeds to step S303.

  In step S303, it is determined whether or not the luminance value Y of the latest frame is greater than or equal to the threshold value YTH for the target coordinates. If the luminance value Y is smaller than the threshold value YTH, the process proceeds to step S304.

  In step S304, for the target coordinates, the luminance value Y of the latest frame plus the luminance average value AveY is output to the image display unit 26 as a new luminance value Y, and the current processing is terminated.

  On the other hand, if it is determined in step S302 that the target coordinate is not a correction target part, or if the luminance value Y of the latest frame is greater than or equal to the threshold YTH for the target coordinate in step S303, the process proceeds to step S305.

  In step S305, image correction (brightness correction) is not performed, and the luminance value Y of the latest frame as it is is output to the image display unit 26 for the target coordinates, and the current processing is ended for the target coordinates.

  When the display cycle of the display 20 is shorter than the imaging cycle of the camera 10, the display 20 displays the same image a plurality of times. For example, when the camera 10 captures an image of 30 frames per second and the display 20 displays an image of 60 frames per second, the display 20 performs image processing on the image captured by the camera 10. (Including those not subjected to image correction) are displayed twice.

  In this way, when the target coordinate is the correction target portion and the luminance value Y of the target coordinate is smaller than the threshold value YTH (turned off), the luminance correction is performed so that the luminance at the time of lighting is obtained, whereby the display 20 Flashing of a headlight or the like displayed on the screen can be prevented. In particular, the turn signal, etc. on the image is appropriately excluded from the correction target using the color data (B value), the blink cycle, etc., so the turn signal, etc., for blinking is displayed blinking. On the other hand, it is possible to prevent blinking display of lighting such as a headlight intended to be constantly lit. In addition, when performing luminance correction, among the past predetermined x frames, the luminance average value AveY at the time of lighting (when the luminance value Y is equal to or greater than the threshold value YTH) is added to the luminance value Y of the latest frame. The brightness can be corrected.

  As mentioned above, although the form for implementing this invention was explained in full detail, this invention is not limited to this specific embodiment, In the range of the summary of this invention described in the claim, various Can be modified or changed.

DESCRIPTION OF SYMBOLS 1 Vehicle periphery monitoring apparatus 10 Camera (imaging means)
20 Display (display means)
21 Image capture unit 22 VRAM
23 Image Analysis Unit 24 DRAM
25 Image processing unit 26 Image display unit

Claims (1)

A vehicle periphery monitoring device comprising: an imaging unit that images a vehicle periphery; and a display unit that is disposed inside the vehicle and displays an image captured by the imaging unit,
The display means includes
The brightness of corresponding parts between images continuously picked up by the imaging means is periodically changing more than a predetermined value, and the corresponding parts of the continuously picked-up images When the luminance change amount of the predetermined color data between the images in which the luminance changes more than the predetermined value is equal to or larger than the predetermined variation amount, or the cycle in which the luminance change of the corresponding part occurs is predetermined. When the period is shorter than the period, the luminance of the corresponding part in the image in which the luminance of the corresponding part is equal to or lower than a predetermined luminance value among the continuously captured images is corrected, and the corrected image is displayed. It is characterized by
Vehicle periphery monitoring device.

Images