JP6204022B2 - Image processing apparatus, imaging apparatus, and image processing method - Google Patents

Description

  The present invention relates to an image processing apparatus, an imaging apparatus, and an image processing method for capturing an image of the periphery of a host vehicle and generating an image for a driver or the like to confirm the periphery.

  2. Description of the Related Art A driving support device that displays a captured image around a vehicle captured by an imaging device mounted on the vehicle on a display device installed in the vehicle when the vehicle is traveling is known. By checking the captured image displayed on the display device, the driver can travel while recognizing obstacles around the vehicle, and can park in the parking area accurately and easily during parking.

JP 2010-1020 A

  However, for example, when the surroundings of the vehicle are illuminated by a light source such as a brake lamp at night and in an indoor space, the entire captured image may be dyed in the color of the light source, which may make it difficult for the driver to see.

  Accordingly, an object of the present invention made in view of such circumstances is to provide an image processing apparatus, an imaging apparatus, and an image processing method that maintain the visibility of a captured image.

In order to solve the above-described problem, an image processing apparatus according to a first aspect of the present invention includes an image processing unit that performs image processing on a captured image obtained by photographing the periphery of a vehicle, and an interface unit that performs communication with the vehicle side. An image processing device that outputs an image after image processing to a vehicle side, wherein the image processing unit applies a red component to the captured image in accordance with information on the brake of the vehicle acquired by the interface unit. It performs a process of reducing the information regarding the brake depression speed of the brake depression amount, or wherein the including that data relating to at least one of the intensity of depression.

  In addition, the image processing unit, when data indicating that the brake depression is fast, the depression amount is large, or the depression is strong is acquired by the interface unit, compared to a case where the data is not acquired. , Greatly reducing the red component.

Moreover, information on the brake includes depression speed of the brake depression amount, or the data of the multi-stage values for at least one of intensity of depression, the image processing unit, the magnitude of the value of the data acquired by said interface unit The amount of red component is reduced according to the above.

  The information on the brake includes data on the level of illuminance of the brake lamp.

Further, the image processing unit, compared with the case data, which means that illumination of brake lights is high when it is acquired by the interface unit, the illumination of the brake lamps data indicating the low equalizer Metropolitan is obtained The red component is greatly reduced.

  In addition, the interface unit acquires a data frame conforming to the CAN communication standard from a predetermined node on the vehicle side, and the image processing unit acquires information related to a vehicle brake from a data field in the data frame. Features.

The image processing unit may transmit a remote frame conforming to a CAN communication standard for requesting the data frame to the vehicle side when acquiring the captured image .

In order to solve the above-described problem, according to a second aspect of the present invention, there is provided an imaging device that outputs a photographed image of the periphery of a vehicle to the vehicle, and the vehicle brake is used when the vehicle periphery is imaged. A remote frame including a signal requesting information on the vehicle is transmitted to the vehicle side by communication conforming to the CAN communication standard, and the information on the brake includes the brake depression speed, the depression amount, or the depression strength. data relating to at least one of and wherein including that.

In order to solve the above-described problem, according to a third aspect of the present invention, there is provided an image processing method in an image processing apparatus, the step of acquiring a captured image obtained by imaging the surroundings of a vehicle, and information relating to vehicle braking. A step of acquiring from the vehicle side, a step of performing image processing for reducing a red component on the captured image based on information about the acquired brake, and a step of outputting the image after the image processing to the vehicle. has information on the brake depression speed of the brake depression amount, or wherein the including that data relating to at least one of the intensity of depression.

According to a fourth aspect of the present invention, in order to solve the above-described problem, there is provided an image processing method, the step of imaging the periphery of the vehicle, and the step of outputting a captured image capturing the periphery of the vehicle to the vehicle. Transmitting a remote frame including a signal for requesting information relating to vehicle brakes to the vehicle side in communication conforming to the CAN communication standard when imaging the vehicle periphery. characterized depression speed of the brake depression amount, or a data including things related to at least one intensity of depression.

  According to the image processing device, the imaging device, and the image processing method according to the present invention configured as described above, it is possible to maintain the visibility of the captured image.

1 is a block diagram showing a schematic configuration of a driving support device including an image processing device according to an embodiment of the present invention. It is the picked-up image which superimposed the driving assistance index displayed on the display part. 5 is a flowchart illustrating processing executed by the image processing apparatus in the embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  First, a driving support device including an image processing device according to an embodiment will be described. As shown in FIG. 1, the driving support device 10 according to the present embodiment includes an imaging unit 11, a brake sensor 12, a lamp switch 13, an ECU (Electronic Control Unit) 14, a display unit 15, and an image processing device 16. Each of them is connected by a communication line. The communication line is a kind of data bus, and performs communication conforming to internationally standardized standards such as CAN, LIN, and Flex Ray as an example.

  The imaging unit 11 is installed so that the rear side of the vehicle can be imaged. For example, the imaging unit 11 is fixed with a depression angle of about 30 ° above the rear part of the vehicle, and images a subject in a region approximately 8 m rearward. The imaging unit 11 includes an optical system and an imaging element, and captures a subject image formed on the imaging element by the optical system. The imaging element has a color filter array and can capture a color image. The imaging unit 11 captures one frame of a subject image every 1/30 seconds, for example, for displaying a real-time moving image. The imaging unit 11 generates a captured image as an image signal by imaging and transmits it to the image processing device 16.

  The brake sensor 12 detects information related to depression of the brake pedal. That is, the brake sensor 12 detects depression information such as the depression speed, depression amount, and depression strength of the brake pedal. The brake sensor 12 outputs the detected depression information to the communication line.

  The lamp switch 13 detects input of turning on and off of the headlamp of the vehicle. The lamp switch 13 outputs the detected input to the communication line.

  The ECU 14 acquires information and inputs detected from various sensors and switches provided in the vehicle via the communication line, and controls various operations of the vehicle. For example, the ECU 14 adjusts the amount of fuel supplied to the engine based on the accelerator opening detected by the accelerator sensor and the temperatures detected by temperature sensors provided in various parts of the engine.

Further, the ECU 14 adjusts the braking force of the brake based on the depression information detected by the brake sensor 12, and lights up a brake lamp that emits red light. In particular, the ECU 14 detects not only the presence / absence of depression, but also the depression speed, depression amount, depression intensity, and the like. ECU14 is detected speed, quantity, while adjusting the braking force of the brake depending on, for example, the intensity, determines the high low illuminance at the time of lighting the brake lamp, performs processing for turning on the brake lamp at the determined illuminance . In other words, rather than early, deeply, when it is detected that depresses the brake pedal strongly, with very strong braking force, to strengthen the lighting of the brake lamp. As a result, when the driver brakes suddenly, it expresses that sudden braking has occurred for the surrounding area including the following vehicle.

Incidentally, processing of Ru enhances the illumination of the brake lamps, not only increases the light emission intensity of the lamp itself, if the brake lamp consists of a number of light emitting elements such as LED, increasing the number of light emitting elements to emit light It may be realized by.

  In these processes, the ECU 14 instructs the brake lamp (or its control device) via a communication line.

  Further, the ECU 14 turns on and off the headlamp based on the lighting input and the lighting off input detected by the lamp switch 13.

  Further, the ECU 14 provides braking information indicating the braking state based on the depression information detected by the brake sensor 12 and lighting information indicating that the headlamp is in the lighting state based on the lighting input detected by the lamp switch 13. Output to the communication line. Here, the braking information may include illuminance information for specifying the illuminance of the brake lamp.

  The display unit 15 is an LCD, for example, and is installed near the driver's seat. The display unit 15 can display a real-time moving image. In order to display a real-time moving image, for example, the display unit 15 acquires an image signal of one frame every 1/30 seconds from the image processing device 16 and switches the image to be displayed every 1/30 seconds. Display moving images.

  The image processing device 16 performs predetermined image processing on the image signal generated by the imaging unit 11 and transmits the image signal to the display unit 15. As described above, the imaging unit 11 generates one frame of image signal every 1/30 seconds, and the image processing device 16 acquires one frame of image signal every 1/30 seconds. The image processing device 16 transmits an image signal obtained and subjected to image processing to the display unit 15 every 1/30 seconds.

  The image processing device 16 has a travel support function for backward travel, and can perform image processing for displaying a travel support index for assisting backward travel on a real-time moving image. The display of the driving support index will be described below together with the configuration of the image processing device 16. The image processing device 16 includes an I / F 17, a normal processing unit 18, a storage unit 19, a detection unit 20, a color determination unit 21, and a synthesis unit 22.

  The I / F 17 is connected to the imaging unit 11, the ECU 14, and the display unit 15, and can communicate information and signals with these devices. For example, the I / F 17 acquires an image signal from the imaging unit 11, and brake information including lighting information or lighting information and lighting information from the ECU 14 via a communication line. The I / F 17 transmits an image signal subjected to image processing in the image processing device 16 to the display unit 15. The I / F 17 is connected to the normal processing unit 18, the storage unit 19, the detection unit 20, the color determination unit 21, and the synthesis unit 22, and communicates information and signals with these parts.

  The normal processing unit 18 performs normal image processing such as AGC, white balance, color interpolation, and gamma correction on the captured image acquired from the imaging unit 11.

  The storage unit 19 stores data necessary for various controls executed by the image processing device 16. For example, the storage unit 19 stores the contour and position of the driving support index.

  Based on the image signal acquired from the imaging unit 11, the detection unit 20 detects a color in the vicinity where the driving support index is superimposed on the captured image.

  The color determination unit 21 adjusts the color of the captured image based on the depression information or the illuminance information included in the braking information and the lighting information. In particular, the color determination unit 21 specifies the illumination intensity of the brake lamp from the depression information, or directly acquires the illumination information to specify the illumination intensity of the brake lamp.

When the brake is in an operating state, that is, when the brake lamp is lit, the color determining unit 21 has a red component in the color of the photographed image as compared with the case where the brake is in an inoperative state (the brake lamp is off). Is adjusted to a slightly reduced color. At this time, adjustment according to the illuminance of the brake lamp specified from the depression information or the illuminance information is performed. That is, the illumination of the brake lamps high have field coupling (sudden braking state), significantly reduced the red component in the captured image, the illumination of the brake lamps low have field coupling (normal brake state), the red in the captured image Ingredients are reduced small.

  Further, the color determination unit 21 refers to lighting information of the headlamps or brightness information of the captured image detected by the detection unit 20 when adjusting the color. When the lighting information can identify that the headlamp is not lit, or when the detection unit 20 can determine that the captured image is brighter than the color determination unit 21, the color determination unit 21 performs a reduction process for lean. The amount to be reduced or reduced may be reduced as a whole.

  The synthesizing unit 22 superimposes the driving support index on the captured image subjected to the normal image processing.

  When the driving support function is executed, as shown in FIG. 2, the captured image 24 on which the driving support index 23 is superimposed is synthesized and displayed on the display unit 15. The driving support index 23 is formed by a vehicle width extension line 25 and a distance guide line 26. The vehicle width extension line 25 is a trajectory that passes through both ends of the vehicle during reverse travel. The distance guide line 26 indicates a predetermined distance from the current vehicle, for example, a distance of 1 m and 3 m. The driver can recognize the traveling direction when the vehicle is moving backward by the driving support index 23.

  Next, the driving support process executed by the image processing device 16 will be described with reference to the flowchart of FIG. The driving support process starts when the input unit included in the image processing device 16 detects an input for starting execution of the driving support function. The driving support process ends when an input for stopping execution of the driving support function is detected.

  In step S <b> 100, the image processing device 16 receives an image signal from the imaging unit 11. When the image signal is received, the process proceeds to step S101.

In step S101, the image processing device 16 determines whether or not the headlamp is in the lighting state from the lighting information, or determines whether or not the brightness of the captured image detected by the detection unit 20 is equal to or greater than a predetermined threshold. That is, in step S101, it is determined whether or not the red reduction process for the captured image is performed. When the headlamp is in the lighting state, there have is determined that the surrounding is dark, the process proceeds to step S102. When the headlamp is not lit, there have is determined that the ambient is sufficiently bright, the process proceeds to step S107.

  In step S102, the image processing device 16 determines whether braking information has been acquired. If the braking information has not been acquired, it is determined that the brake lamp is off, and the process proceeds to step S106.

In step S103, the image processing device 16 specifies the illuminance of the brake lamp from the depression information or the illuminance information. Here, for the sake of simplicity of explanation, the illuminance of the brake lamp is simply set to a binary value of high / low, and when the illuminance of the brake lamp is high , the process proceeds to step S104. Illumination of the brake lamps in the lower have field coupling, the process proceeds to step S105.

In step S104, the image processing apparatus 16, the illumination of the brake lamps high have field coupling (sudden braking state), greatly reducing the red component in the captured image. Thereafter, the process proceeds to step S106.

In step S105, the image processing apparatus 16, the illumination of the brake lamps in the lower have field coupling (normal brake state), slightly reduce the red component in the captured image. Thereafter, the process proceeds to step S106.

  In step S <b> 106, the image processing device 16 superimposes the driving support index 23 on the captured image 24. After the driving support index 23 is superimposed on the captured image, the process proceeds to step S108.

  In step S <b> 107, the image processing device 16 transmits the captured image 24 on which the driving support index 23 is superimposed to the display unit 15. After transmitting the captured image 24, the process returns to step S100.

  According to the image processing device 16 of the embodiment configured as described above, when the travel support index 23 is drawn, the color of the captured image serving as the background is adjusted. As described above, the image processing device 16 performs image processing in consideration of the influence on the captured image due to the light emission of the brake lamp. In particular, referring to the illuminance of the brake lamp caused by how to depress the brake pedal, in the case of depressing the brake pedal that performs sudden braking, the red color is greatly reduced from the shot image, and in the case of depressing the brake that is not sudden braking Reduce the red color small. As a result, it is possible to always perform a display that is not easily affected by how the brake pedal is depressed, and the visibility can be further improved.

  Further, according to the image processing apparatus 16 of the embodiment, the red color reduction process is performed only when the headlamp is turned on. When the headlamps are extinguished, it is considered that the outside light is bright enough during the daytime. Therefore, in such a situation, even if the brake lamp is lit, it is considered that the color of the subject around the vehicle is not easily affected by the red color, and the driver can visually recognize without being affected by the brake lamp.

  Such determination of ambient brightness by the image processing device 16 may be performed not by determining whether the headlamp is turned off, but by determining brightness from the captured image itself by the detection unit 20. Thus, when it is determined that the surroundings are sufficiently bright, the image processing device 16 performs display without performing special red reduction processing. As a result, even if the brake lamp is lit, the captured image 24 is hardly affected. Therefore, it is possible to avoid the situation where the redness is reduced unnecessarily and the visibility of the captured image is deteriorated.

Further, as an example embodiment, when identifying the illumination of brake lights from depression information or the illuminance information, for simplicity of explanation, it has been described by example of using only the high low binary to binary It is not limited, It is good also as multi-stage values, such as 3 levels of high , medium, and low , or 10 steps.

  A process when multi-stage values are used will be described.

  First, the brake sensor 12 digitizes the stepping speed, amount, and strength, for example, in 10 steps, and outputs this as stepping information to the ECU 14 via a communication line. Then, the ECU 14 specifies the level of braking based on the depression information and digitizes it in, for example, 10 steps. Further, the ECU 14 quantifies the illuminance of the brake lamp in 10 steps corresponding to the level of braking. The ECU 14 outputs numerical information on the braking level and brake lamp illuminance to the communication line as braking information.

On the other hand the image processing apparatus 16, as the processing corresponding to step S103 of FIG. 3, to detect the value of illuminance rather than high low illuminance, stepwise red depending on the illuminance high low instead of steps S104, S105 Reduction processing is performed.

  Next, the flow when the image processing device 16 acquires the braking level or the numerical information of the brake lamp illuminance from the ECU 14 will be described by taking as an example a case where a communication line conforming to the CAN communication standard is used.

  In the CAN communication standard, information exchange between nodes is performed by exchanging information called a data frame and a remote frame between the nodes.

  The data frame includes an ID (Identifier) field for identifying data contents and a transmission node, a data field that is actual transmission data, and a CRC (Cyclic Redundancy Check) field for determining whether data has been received normally. . The remote frame has a basic frame structure in which a data field is removed from a data frame. The node that has received the remote frame returns necessary information regarding the type of data specified by the ID of the remote frame in the data field, and then returns the data frame to the transmission source node.

  Specifically, when imaging is performed by the imaging unit 11 and a captured image is output, the image processing unit 16 causes the interface unit 17 to transmit a remote frame to the ECU 14. This remote frame is provided with an ID intended to request information related to braking (braking information or brake lamp illuminance information), and the transmission destination node is configured as an ECU 14.

  When the ECU 14 receives the remote frame, the ECU 14 determines a request for information on the brake from the ID, generates a data frame including the braking information or the illuminance information of the brake lamp in the data field, and the destination node as the image processing unit 16. Output to the communication line.

  When the image processing unit 16 receives this data frame, the braking information or the illuminance information of the brake lamp described in the data field is extracted, and the subsequent red reduction processing is performed. The above is an example in which the CAN communication standard is used for the communication line, but standards such as Flex Ray and LIN may be used in the same manner. In that case, when performing communication processing for acquiring numerical information of brake lamp illuminance via the communication line, data compliant with the respective standards is transmitted and received between the ECU 14 and the image processing unit 16. As described above, by making the interface unit 17 correspond to the standardized standard, it is possible to realize the versatile image processing apparatus 16 that can support various vehicles.

  Although the present invention has been described based on the drawings and embodiments, it should be noted that those skilled in the art can easily make various changes and modifications based on the present disclosure. Therefore, it should be noted that these variations and modifications are included in the scope of the present invention.

DESCRIPTION OF SYMBOLS 10 Driving assistance apparatus 11 Imaging part 12 Brake sensor 13 Lamp switch 14 ECU (Electronic Control Unit)
15 Display Unit 16 Image Processing Device 17 I / F
18 Normal Processing Unit 19 Storage Unit 20 Detection Unit 21 Color Determination Unit 22 Composition Unit 23 Driving Support Index 24 Captured Image 25 Vehicle Width Extension Line 26 Distance Reference Line

Claims (10)

An image processing device that includes an image processing unit that performs image processing on a captured image obtained by photographing the periphery of a vehicle, and an interface unit that communicates with the vehicle side, and outputs the image processed image to the vehicle side,
The image processing unit performs processing for reducing a red component on the captured image according to information on the brake of the vehicle acquired by the interface unit,
Information on the brakes, the image processing apparatus according to claim depression speed of the brake depression amount, or a data including things related to at least one intensity of depression. The image processing unit is larger when data indicating that the brake is depressed quickly, a large amount of depression, or a strong depression is acquired by the interface unit than when the data is not acquired. The image processing apparatus according to claim 1, wherein a red component is reduced. The information on the brake includes multi-stage data regarding at least one of the brake depression speed, the depression amount, and the depression intensity,
The image processing apparatus according to claim 1, wherein the image processing unit performs a red component reduction process in an amount corresponding to a magnitude of a data value acquired by the interface unit. The image processing apparatus according to claim 1, wherein the information regarding the brake includes data regarding the level of illuminance of the brake lamp. In the case where data indicating that the illuminance of the brake lamp is high is acquired by the interface unit, the image processing unit is compared with the case where data indicating that the illuminance of the brake lamp is low is acquired, The image processing apparatus according to claim 4, wherein the red component is greatly reduced. The interface unit acquires a data frame conforming to the CAN communication standard from a predetermined node on the vehicle side,
The image processing device according to any one of claims 1 to 5 , wherein the image processing unit acquires information relating to vehicle braking from a data field in the data frame. The image processing apparatus according to claim 6, wherein the image processing unit transmits a remote frame conforming to a CAN communication standard for requesting the data frame to the vehicle side when acquiring the captured image. . An imaging device that outputs to the vehicle a photographed image of the surroundings of the vehicle,
When imaging the periphery of the vehicle, an interface unit that transmits a remote frame including a signal requesting information on the brake of the vehicle to the vehicle side by communication conforming to the CAN communication standard is provided.
Information relating to the brake, an imaging apparatus, wherein the depression speed of the brake depression amount, or including that data relating to at least one of the intensity of depression. An image processing method in an image processing apparatus,
Obtaining a captured image of the surroundings of the vehicle;
Obtaining information about vehicle brakes from the vehicle side;
Performing image processing for reducing the red component based on the acquired brake-related information on the captured image;
Outputting the image after the image processing to the vehicle,
It said information on brakes, an image processing method in an image processing apparatus according to claim depression speed of the brake depression amount, or a data including things related to at least one intensity of depression. A images processing method,
Imaging the periphery of the vehicle;
Outputting a photographed image of the surroundings of the vehicle to the vehicle;
Transmitting a remote frame including a signal for requesting information on the brake of the vehicle to the vehicle side in communication conforming to the CAN communication standard when imaging the periphery of the vehicle,
Said information on brakes, image processing method, wherein the depression speed of the brake depression amount, or a data including things related to at least one intensity of depression.

Images