WO2023243162A1

WO2023243162A1 - Camera control device and vehicle control device

Info

Publication number: WO2023243162A1
Application number: PCT/JP2023/008918
Authority: WO
Inventors: 哲也山田; 宏治土井; 聡松田; 翔太郡司
Original assignee: 日立Astemo株式会社
Priority date: 2022-06-17
Filing date: 2023-03-08
Publication date: 2023-12-21

Abstract

Provided are a camera control device and a vehicle control device that can be realized by turning headlights off for a short time and comparing images when light is turned off and turned on in order to detect a reflective object using a camera and use the reflective object for object detection in driving assistance, and that, due to the camera and the headlights being physically separated, make it possible to asynchronously control the imaging timing of the camera and the turning-on/-off of the headlights as well as to capture images near the timing of switching between turning on and turning off. The time phase of an imaging timing pattern (imaging phase) of a camera imaging unit 2 is adjusted, as a result of which an imaging time interval at the time of switching between turn-off and turn-on of a headlight unit 10 is curtailed (shortened).

Description

Camera control device and vehicle control device

The present invention improves recognition accuracy at night by utilizing headlight control by the vehicle's own camera at night in a vehicle that has a camera, headlights, and advanced driver assistance system (ADAS) functions. The present invention relates to a camera control device and a vehicle control device that can appropriately provide driving support using this recognition.

When driving at night, the surroundings are dark, so visibility is low. For example, drivers may be slower to spot danger from pedestrians or unlit bicycles than during the day. When a reflective object is attached to a pedestrian or bicycle, the headlights of the own vehicle are reflected on the reflective object, which serves as an auxiliary means for drivers to prevent delays in detecting danger.

A method has been proposed for a vehicle system that is equipped with a camera and uses information from the camera to control the illumination of headlights to assist the driver in driving at night.

Light distribution control is a technology that has been proposed for a long time. For example, as in Patent Document 1, the distance between the vehicle and the preceding vehicle is measured, and the high beam and low beam are switched depending on the distance between the vehicles at night. Various proposals have been made as conditions for switching headlight control, including not only the presence of a preceding vehicle but also the presence or absence of an oncoming vehicle.

To determine whether an object is a reflective object or a luminescent object, turn off your vehicle's headlights for a short time. Reflective objects disappear when your vehicle's headlights are turned off, and luminescent objects do not disappear when the vehicle's headlights are turned off. By turning off the headlights, reflective objects and light-emitting objects can be distinguished.

Patent Document 2 proposes a technique that removes reflective objects and facilitates the detection of oncoming vehicle lights. The headlights and camera are controlled in synchronization at the same time, and the camera captures images only at the same time as the moment the headlights are turned off, making it possible to remove reflective objects. As a result, the image captured by the camera is only of the light-emitting object, making it easier to detect the lights of an oncoming vehicle.

Next, in order to detect reflective objects and light-emitting objects, Patent Document 3 discloses that headlights and cameras are controlled in synchronization at the same time, and images are always taken alternately when the headlights are turned off and turned on. , which realizes the detection of reflective objects and luminescent objects. Furthermore, by dividing the irradiation area of the headlight and performing light control independently for each area, for example, it is possible to lengthen the lights-out period only in a specific area to prevent dazzling. On the other hand, since the timing of imaging corresponds to the timing of turning off and lighting up the entire area, the longer the off time of a specific area becomes, the longer the interval between imaging becomes.

As described above, it has been proposed to perform headlight control in combination with the removal of reflective objects or the detection of reflective objects.

Japanese Patent Application Publication No. 6-84099 Japanese Patent Application Publication No. 2007-76429 Japanese Patent Application Publication No. 2011-110999

When detecting a reflective object by controlling the illumination of headlights, it is desirable that the interval between the imaging timings (imaging time interval) between when the headlights are turned on and when they are turned off is as short as possible. When comparing images when the lights are on and images when the lights are off, if the time interval between image acquisitions is long, the objects in the images may move a lot and disappear from the screen, and objects that are reflected may disappear due to reflective objects. It is necessary to determine whether the object has disappeared from the screen due to movement.

If the time interval between the imaging timings of turning on and off is shortened, it becomes difficult to synchronize and control the camera imaging timing and the headlight illumination control at the same time. If the camera and headlight are independent devices and are physically separated and the information between the devices is communicated via CAN, there will be processing delays in each device's microcontroller and communication delays between the devices via CAN. Therefore, a delay of several tens of ms or more is required, and synchronous control is considered difficult. For this reason, it is necessary to control the camera's imaging timing and the headlight illumination control asynchronously. In both Patent Documents 2 and 3, the imaging timing of the camera and the illumination control of the headlights are synchronized, so they are not suitable for imaging at short time intervals between turning on and turning off the lights.

Furthermore, in order to use the camera not only for light distribution control but also for object detection for use in driving support such as automatic emergency braking, it is necessary to repeatedly capture images within a specific cycle, such as within a 50 ms cycle. Capturing images only when the lights are turned off as in Patent Document 2, or extending the lights-out period and increasing the imaging interval to prevent dazzling as in Patent Document 3 are not suitable for imaging for driving support. .

Therefore, in order to detect reflective objects with a camera and use it for object detection for driving support, it is necessary to shorten the imaging timing when the lights are turned on and off, and to control the camera imaging and the headlights on and off asynchronously. The challenge is to provide the following.

Therefore, an object of the present invention is to provide a camera control device and a vehicle control device that have means for solving this problem. In order to shorten the imaging timing when the lights are turned on and off, and to control the camera imaging and the headlights on and off asynchronously, (1) a variable timing control means that makes the timing of camera imaging variable; (2) the headlights. (3) Illumination control means using a duty ratio as a means for controlling turning on and off of the headlights; (3) aligning the timing phases because the imaging timing and the headlight irradiation control timing are asynchronous to shorten the imaging timing when turning on and off. It is an object of the present invention to provide a camera control device having three means: a control means for controlling the phase of the camera;

Furthermore, in the present invention, since reflective objects such as pedestrians and bicycles can be detected by the above three means, (4) automatic emergency braking can be performed using the detection information of reflective objects. It is an object of the present invention to provide a vehicle control device having driving support means such as the following.

In order to achieve the above object, a camera control device according to the present invention is a camera control device that controls a headlight section that illuminates the front of the own vehicle, and a camera imaging section that takes an image of the front of the own vehicle, The control device sets a lighting/extinguishing pattern of a predetermined frequency and duty in the headlight section, controls the headlight section to turn on/off using the lighting/extinguishing pattern, and sets the headlight section to turn on/off using the lighting/extinguishing pattern. Based on a predetermined frequency and duty for the headlight section, an imaging timing pattern different from the lighting/extinguishing pattern of a regular first imaging time interval and a second imaging time interval shorter than the first imaging time interval. By setting , and adjusting the time phase of the imaging timing pattern of the camera imaging section, the imaging time interval when switching between turning off and lighting the headlight section is reduced.

Furthermore, when the reflective object determined to be a stationary object is located at a certain distance, the vehicle control device according to the present invention determines that it is a road boundary, and issues an alarm or alert to prevent intrusion into the road boundary. Controlling the steering of the host vehicle. In addition, when the reflective object is the moving object, the vehicle control device according to the present invention determines that the moving object is a pedestrian or a bicycle based on a comparison with the relative speed of the own vehicle, and The system predicts the future trajectory from the vehicle, calculates the risk of collision with the own vehicle, and issues a warning or performs automatic emergency braking control when it is determined that the risk of collision with the own vehicle is high.

As described above, according to the present invention, reflective objects and light-emitting objects can be detected by periodically turning off the headlights for short periods of time and capturing images at short timings at the boundary between turning off and turning on the headlights. Among objects, by correctly removing pseudo-objects that appear on the road due to headlight reflections, etc., it is possible to avoid incorrect headlight light distribution control or automatic emergency braking caused by pseudo-objects. Furthermore, various advanced driving assistance systems (Advanced Driving Assistance Systems) are being developed, such as using reflective objects such as curbs and guardrails as road boundaries to improve the performance of automatic departure avoidance systems by using them as substitutes when lane detection is not possible. Driver Assistance Systems (ADAS) can be realized.

Problems, configurations, and effects other than those described above will be made clear by the description of the embodiments below.

FIG. 1 is a system configuration diagram in a first embodiment of the present invention. An example of a screen display of a reflective object and a light emitting object on a curve in the first embodiment of the present invention, (a) is an image when the light is turned on at time t1, and (b) is an image when the light is turned off at time t1. , (c) is an example of an image when the lights are turned off at time t2. It is an example of the movement vector in the image at the time of lighting in the first example of the present invention, where (a) is the image when the lighting is on at time t1, and (b) is an example of the image when the lighting is on at time t2. 2 is a timing chart of lights and imaging in the first embodiment of the present invention, (a) is a timing chart before phase adjustment of headlight on/off and imaging timing, and (b) is a timing chart of headlight on/off and imaging timing. Timing chart after phase adjustment. 5 is a flowchart of phase adjustment of light on/off and imaging timing in the first embodiment of the present invention. 1 is a flowchart of object recognition in the first embodiment of the present invention. 1 is a flowchart of lane/road boundary recognition in the first embodiment of the present invention. 1 is a flowchart of light distribution control in the first embodiment of the present invention. 1 is a flowchart of automatic emergency brake control in the first embodiment of the present invention. 1 is a flowchart of automatic lane deviation control in the first embodiment of the present invention. 5 is a timing chart of lighting and imaging in a second embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described based on the drawings.

[First example]
In the first embodiment of the present invention, in order to detect reflective objects with a camera and use the object detection for driving support, the headlights are turned off for a short period of time, and the imaging timing when the lights are turned on and off is shortened. In response to the problem of asynchronously controlling camera imaging and headlight on/off, we have developed (1) variable timing control means for making the camera imaging timing variable; and (2) means for controlling headlight on/off. (3) phase control means for shifting the phase of the imaging timing in order to adjust the imaging timing for shortening the imaging timing when the lights are turned on and off and the illumination control timing of the headlights; and (4) providing means for driving support such as automatic emergency braking using the detection information of reflective objects.

Prior to explaining the configuration of the vehicle system of the first embodiment, the principle of the first embodiment will be explained using FIGS. 2 to 4.

Figure 2 shows an example of a screen display of a reflective object and a luminous object on a curve. The car is heading towards a curve. FIG. 2(a) shows an image when the headlights are illuminated at time t1. The light-emitting object is a headlight 100 of an oncoming vehicle. The reflective objects include a curb reflective object 101 and a guardrail reflective object 102, which are lined up at regular intervals. When the visibility of road boundaries is low, the approximate location of road boundaries can be determined by using these reflective objects lined up at regular intervals. There is also an object 103 that reflects when the headlights of the own vehicle are turned on. This reflective object 103 may be reflected by a reflector of a person or a bicycle, but it may also be a pseudo object that is visible on the road due to the reflection of headlights or the like. FIG. 2B shows an image when the headlights were turned off at the same time t1, in which the reflective object is not displayed on the screen, and only the light-emitting object, the headlight 100 of the oncoming vehicle, is displayed on the screen. By comparing the lights-on image (FIG. 2(a)) and the lights-off image (FIG. 2(b)), not only curbs and guardrails but also other reflective objects 103 can be separated. FIG. 2C shows an image when the headlights are turned off at time t2, which is after time t1. At this time, the own vehicle approaches the curve for a period of time t2-t1, and the oncoming vehicle heads toward the own vehicle from the curve, so the headlights 104 of the oncoming vehicle appear at the bottom of the screen compared to FIG. 2(b). The position will be Furthermore, as time passes, the oncoming vehicle is located outside the screen, so the headlights of the oncoming vehicle, which are light-emitting objects, disappear from the screen.

Since there is a time difference between the turning on and turning off of the headlights, Figs. 2(a) and 2(b) will never occur at the same time. The image when the lights are turned on is shown at time t1, and the image when the lights are turned off is shown in Figure 2(c) when the time t1 is a different time and the time after t1 is set to t2.If the interval between times t2 and t1 is long, the environmental conditions are When comparing a lit image and an unlit image, it is preferable that this time interval be short, since this increases the possibility that the light-emitting object will disappear from the screen.

In FIG. 2, it is possible to determine whether an object is a reflective object because it is not detected in the image when the lights are off, but FIG. 3 shows the determination of moving objects, stationary objects, and pseudo objects among reflective objects.

FIG. 3 shows an example of detection of a moving object, a stationary object, and a pseudo object as reflective objects. It is assumed that the moving object of the reflective object is a bicycle, and a reflective object is affixed somewhere on the bicycle, such as the rear wheel. The stationary reflective object is assumed to be a reflective object affixed to a telephone pole or the like. Pseudo objects refer to objects that appear to be objects, such as reflections of one's own vehicle or the reflection of one's own vehicle's headlights on the road, even though they are not actually objects.

It is assumed that at time t1, the headlights are turned on, and a reflective object 111 and a reflective pseudo object 113 on the inside of the roadway, and a reflective object 112 on the outside of the roadway are detected (FIG. 3(a)). It is assumed that the headlights continue to be turned on at time t2 after time t1, and a reflective object 114 and a reflective pseudo object 116 on the inside of the roadway, and a reflective object 115 on the outside of the roadway are detected (FIG. 3(b)). By comparing the images at time t1 and time t2, the vector in the moving direction can be calculated from the position of the reflecting object. The movement vector of the reflective object inside the roadway is 117, and the movement vector of the reflective object outside the roadway is 118. Since the reflective pseudo object 116 is always at the same location on the screen, the movement vector 119 is zero (FIG. 3(b)).

As a result, pseudo objects can be identified due to the absence of movement vectors. Further, stationary objects and moving objects can be distinguished from the direction and magnitude of the movement vector.

FIG. 4 shows the principle of imaging at variable timing when the headlights are turned on and off in the first embodiment of the present invention.

FIG. 4(a) shows a timing chart before phase adjustment of headlight on/off and imaging timing, and FIG. 4(b) shows a timing chart after phase adjustment.

First, the camera issues an instruction to the headlights regarding the duty ratio of turning them off and on. The headlights receive the duty ratio instruction, and the headlights repeatedly turn off and on according to the duty ratio. In FIG. 4, the duty ratio between turning off and turning on is 1:9, so turning off and turning on is repeated at a time interval of 9 when turning on for 1 when turning off.

The camera has a shutter control that allows variable imaging timing, and performs imaging at short time intervals and imaging at long time intervals. The camera shutter control time interval repeats periodically, independently and asynchronously of the headlights. In FIG. 4A, there are short time intervals between times t1 and t2 and between t2 and t3, and long time intervals between t3 and t4 and between t4 and t5. Capturing images when the headlights are off produces a dark image, while capturing images when the headlights are on produces a bright image. In FIG. 4A, time t4, time t9, and time t14 are images when the headlights are turned off, and images at other times are images when the headlights are turned on.

To shorten the time interval between when the lights are off and when the lights are on for image comparison for determining reflective objects, and to shorten the time interval when the lights are off in order to obtain multiple images when the lights are off. It is necessary to match the short point with the timing of turning off the headlights. Since turning on/off of the headlights and the imaging timing are both cyclical processes, by adjusting the phase of the imaging timing, it is possible to match a portion with a short imaging interval to the timing of turning off the headlights.

FIG. 4(b) shows a timing chart after phase adjustment of headlight on/off and imaging timing. Time t1, time t2, time t6, time t7, time t11, and time t12 are images when the headlights are turned off, and images at other times are images when the headlights are turned on. In FIG. 4(a), imaging is performed once when the lights are off, whereas in FIG. 4(b), imaging can be performed twice when the lights are off. Furthermore, it is possible to capture an image in the vicinity of the change in headlight turning off and turning on. In this way, by adjusting the phase of the asynchronous but periodic headlight on/off timing and imaging timing, many images are taken when the headlights are off and near the change between the headlights off and on, and at regular intervals when the headlights are on. can be imaged with.

That is, the on/off duty of the headlights, the corresponding variable imaging timing of the camera, and the number of images to be acquired when the lights are off and when the lights are on are determined in advance, for example, 2 images when the lights are off and 6 images when the lights are on. In Fig. 4(a), there are 1 image when the lights are off and 7 images when the lights are on.Since the number of images is different, by adjusting the phase of the imaging timing, in Fig. 4(b), there are 2 images when the lights are off and 7 images when the lights are on. Since the number of images is six, which is the same as the predetermined number of images when the lights are off and images when the lights are on, the state is after phase adjustment. Self-adjustment (self-test) of phase adjustment can be performed in this manner.

FIG. 1 shows an example of the configuration of a vehicle system according to the first embodiment of the present invention, which detects reflective objects with a camera and is used for object detection for driving support. The camera is not particularly limited to a monocular camera or a stereo camera, but in the first embodiment, it is a stereo camera that has two left and right lenses and can detect the distance of an object by trigonometry.

The vehicle 1 includes a camera imaging unit 2 that has a built-in camera lens and an image sensor and captures images in front of the vehicle, a camera control unit 3 that processes captured images and detects and recognizes objects, and uses recognition information. A vehicle control section 4 performs driving support such as brake control and steering control, an illuminance meter 5 that indicates illuminance outside the vehicle, a CAN 6 that performs in-vehicle communication, and a headlight section 10 that controls headlights that illuminate the front of the own vehicle. , a map section 11 that has information on the location of the vehicle and its surrounding environment, a steering section 12 and a brake section 13 that are actuators of the vehicle.

The camera imaging unit 2 has a function of capturing an image, captures an image, and outputs a RAW image via an LVDS cable. Although not shown in the figure, it includes left and right lenses, an image sensor, a serializer for converting image digital data into a serial signal, and a shutter setting section 27. The shutter setting unit 27 stores timing information of the imaging shutter. For example, information (imaging timing pattern) such as a cycle of 350 ms, a total of 8 shutters, 2 for 25 ms, and 6 for 50 ms is stored (see FIG. 4).

The camera control unit 3 is connected to the camera imaging unit 2 via an LVDS cable, and includes an image processing unit 20 that processes the transferred RAW image, a recognition unit 21 that recognizes objects and lanes on the image, and a recognition unit 21 that controls turning on and off the headlights. It is comprised of a light control section 23 that instructs light distribution control to switch between high beam and low beam depending on timing and the presence or absence of a vehicle, and a CAN-IF section 22 that is connected to a CAN bus and sends and receives data.

The image processing unit 20 has a function of instructing the camera imaging unit 2 to take a shutter, which is the timing of image capture, and a function of generating an image necessary for recognition from the transferred RAW image. The former includes an imaging timing calculation unit 30 that calculates the imaging timing of the camera imaging unit 2, shifts the phase of the imaging timing to match the turning on and off of the light, and issues a shutter instruction to the camera imaging unit 2, which is the imaging timing. This is performed by the imaging phase adjustment section 31. Although the latter is omitted in the figure, a deserializer that generates RAW image data from a serial signal, a parallax image generation unit 33 that generates a parallax image having distance information in pixel units from the left and right RAW images, and one of the left and right RAW images. It is composed of a brightness image generation section 32 that generates a brightness image from a RAW image, an image comparison section 34 that compares two or more images with a time difference, and an edge image generation section 35 that generates an edge image from the gradient of the brightness image.

The recognition unit 21 includes a lights-off duty calculation unit 40 that determines the duty of the time when the headlights are turned on and off, a pseudo object removal unit 41 that removes pseudo objects that are objects that do not actually exist, such as the reflection of the own vehicle's lights, and a stationary object removal unit 41 a moving object determination unit 42 that determines whether a vehicle is a moving object; a lane/road boundary detection unit 43 that detects lanes from images and detects road boundaries using reflective objects such as lanes, curbs, road shoulders, or guardrails; and light distribution control. The light distribution light detection section 44 detects the headlights and taillights of other vehicles.

The light control unit 23 includes a duty instruction unit 70 for determining the timing of turning on and off the headlights using a duty ratio, and a light distribution instruction unit 71 for instructing light distribution control to switch between high beam and low beam depending on the presence or absence of a vehicle.

The vehicle control unit 4 includes a lane departure control unit 24 that determines whether the vehicle deviates from the lane and issues a warning or a steering control instruction to return the vehicle to the lane when the vehicle deviates, and a lane departure control unit 24 that determines whether the vehicle will collide with an obstacle. It is comprised of an automatic emergency brake control section 25 that determines whether a collision is occurring and automatically issues an emergency braking instruction when there is a high possibility of a collision, and a CAN-IF section 26 that is connected to the CAN bus and sends and receives data.

The lane deviation control unit 24 includes a vehicle trajectory calculation unit 52 that predicts and calculates the vehicle trajectory, a deviation determination unit 53 that determines whether the vehicle will deviate to the road boundary from the vehicle trajectory and road boundary information, and a lane departure determination unit 53 that determines whether the vehicle will deviate to the road boundary. The control unit 51 includes a control unit 51 that issues an alarm or a steering control instruction when the vehicle is in use.

The automatic emergency brake control unit 25 includes an obstacle trajectory calculation unit 60 that calculates the trajectory of an obstacle, a risk calculation unit 61 that calculates the trajectory of the vehicle and the risk of collision with an obstacle, and a risk calculation unit 61 that calculates the trajectory of the own vehicle and the risk of collision with the obstacle. The control unit 62 is configured to generate a warning or brake instruction based on the information provided.

In this example, turning on and off the light and taking images with the camera operate independently and asynchronously, but both are cyclic processes, and by adjusting the phase, a specific number of images can be taken when the light is on and when the light is off. I do. Although the light is an example of automatic lighting, the present invention is not limited and can also be applied to manual lighting.

FIG. 5 shows a flowchart for phase adjustment of light on/off and imaging timing.

First, when controlling the turning on and off of the light, the settings are started when the brightness level on the illuminance meter 5 decreases. The illumination meter 5 calculates the illuminance and transmits the illuminance to the camera control unit 3 (S11). When the illuminance is not below the threshold value (S12), the control for turning on and off the lights is not started, and when the illuminance is below the threshold value (S12), the control for turning on and off the lights is started. The lights-off duty calculation unit 40 calculates the lights-off duty using the data held in the memory (S13). The lights-out duty indicates the cycle time (time interval between lights-out and lights-on) and the time ratio between lights-out and lights-on. Using the light-off duty, the image processing section 20 and the light control section 23 each perform the following. In the image processing section 20, the imaging timing calculation section 30 uses the lights-off duty to calculate the shutter timing for imaging how many times to take images when the lights are off and when the lights are on, and instructs the shutter timing to the camera imaging section 2 ( S14). When the lights are off, many images are taken in a short period of time, when the lights are on, images are taken at regular intervals (regularly), and the shutter timing is calculated so that the images can be taken near the switching between the lights off and the lights on. That is, the imaging timing calculation unit 30 causes the camera imaging unit 2 to perform dots at regular intervals and at intervals shorter than the intervals based on a lighting/extinguishing pattern of a predetermined frequency and duty for the headlight unit 10, which will be described later. Set an imaging timing pattern that is different from the lights-off pattern. The camera imaging unit 2 sets the shutter timing in the shutter setting unit 27 based on the shutter timing (imaging timing pattern) transferred from the image processing unit 20, and performs imaging at the set timing (S15). In parallel with the image processing section 20, the light control section 23 receives the lights-off duty and instructs the headlight section 10 to set the lights-off duty using the duty instruction section 70 (S16). That is, the duty instruction section 70 sets a predetermined frequency (corresponding to the time interval between turning off and turning on) and a turning-on/off pattern of duty in the headlight section 10. Further, the light distribution instruction section 71 uses the light on/off pattern to control (the light distribution of) the headlight section 10 to turn on/off. The headlight section 10 sets a light-off duty (light-on/off pattern) in the light-off duty setting section 28, and periodically turns on and off at the light-off duty determined by the light distribution control section 29 (S17). For example, if the cycle is 1 sec and the light-off duty is 1:9, the light will be turned on and off repeatedly with a light-off time of 100 ms and a lighting time of 900 ms (see FIG. 4(a)).

Next, in order to adjust the phase of the headlight on/off and the imaging timing, the imaging phase adjustment section 31 performs phase adjustment. The imaging phase adjustment unit 31 shifts the phase of the shutter timing (imaging timing pattern) and instructs the camera imaging unit 2 to provide phase information (S18). The camera imaging unit 2 sets phase information in the shutter setting unit 27, and performs imaging by shifting the phase at the set timing (S19). The imaging phase adjustment unit 31 measures the brightness of a specific pixel on each screen (for example, a pixel in a location that is not directly illuminated by the headlights but always makes a difference when the headlights are turned on or off), and uses that brightness to determine the brightness when the headlights are turned on or off. Determine whether it is an image or an image when the lights are off, and check whether the number of periodic images when the lights are on and the number of images when the lights are off are as expected (in other words, whether the number of images when the lights are on periodically and the number of images when the lights are off are as expected with the shutter timing in the relevant phase (S20). If the number of images when the light is on is different from the number of images when the light is off (S21), the imaging phase adjustment unit 31 performs phase adjustment in order to perform phase adjustment again (S18). In this way, it has a self-adjustment (self-test) function for phase adjustment.

After automatic phase adjustment is performed when the light starts turning on (see FIG. 4(b)), recognition processing is performed using obstacles (objects) on the screen.

Figure 6 shows a flowchart of object recognition. The camera is assumed to be a stereo camera, but is not particularly limited.

First, the image processing unit 20 of the camera control unit 3 receives left and right RAW images from the camera imaging unit 2 (S31). In order to extract the brightness of the object, the brightness image generation unit 32 generates a brightness image from the right RAW image (S32). Either the left or right RAW image may be used as the brightness image. The imaging phase adjustment unit 31 measures the brightness of a specific pixel on each screen, for example, a pixel in a location that is not directly illuminated by headlights but always shows a difference when the headlights are turned on or off. It is determined whether the headlights are on or off based on a predetermined luminance threshold, and an on/off attribute indicating whether the headlights are on or off is calculated for the luminance image (S33). Further, the parallax image generation unit 33 uses trigonometry from the left and right RAW images to generate a parallax image having distances for each pixel (S34). The parallax image generation unit 33 adds an ID as an identification number to each detected object using the parallax image, and adds relative position attributes including distance from the own vehicle (S35 ).

Next, the image comparison unit 34 determines whether the object is a reflective object or a luminescent object, and adds reflective/luminous object attributes. The attributes will be updated as needed until they are finalized. Since the object that can be detected in the illuminated image may be both a luminescent object and a reflective object, it is set to an indeterminate attribute and is set to be undetermined. Objects that can be detected in the unlit image are determined to be luminescent objects. Since the on/off attribute is added to the image, when one of the images taken at a certain time (t1) and the image taken at the previous time (t2) has the on attribute and the other has the off attribute, the image Make a comparison. The brightness images when the lights are on and off are compared, and objects that are detected both when the lights are off and when the lights are on are determined (extracted) as luminescent objects, and objects that are detected only when the lights are on are determined (extracted) as reflective objects. The types of these luminous objects and reflective objects are added as reflective/luminous object attributes (S36).

Further, the image comparison unit 34 compares a plurality of brightness images taken at consecutive times, calculates a movement vector from the difference in relative coordinates of the object, and adds it (S37). Objects include both reflective objects and luminescent objects. When comparing the luminance images of two images, there are three types: between lights on, between lights off, and between lights on and off.When comparing between lights on, movement vectors can be calculated for both luminescent objects and reflective objects, but when comparing lights between lights, and between lights on and lights off. In this case, the movement vector of only the light-emitting object can be calculated. By using a plurality of trajectory images taken at consecutive times, it is impossible to detect the unlit image portion of the reflective object, so the movement vector is calculated by complementing and connecting the brightness images when the reflecting object is on. Each object holds as an attribute a movement vector between a certain time (t1) at which the image was taken and a time (t2) immediately before the image was taken. For a reflective object whose light is off at time (t1) and turned on at time (t2), the light is on at the time (t3) immediately before the time (t2), so the time (t2) and the time The relative position at time (t1) is predicted using the relative position at (t3). Since the location of time (t1) is unknown, we assume that there is a uniform linear motion from time (t2) and time (t3), and assume the relative coordinates of time (t1). ) Calculate the movement vector from the relative coordinates of Here, an example is shown in which time (t3) is used, but in order to improve accuracy, a plurality of coordinates of imaging times further back in time may be used, which improves prediction accuracy.

Among the reflective objects, for those that are not actually objects but appear to be objects due to the influence of the own vehicle's headlights, the pseudo object removal unit 41 removes them from among the objects that have reflective/luminous object attributes. An object whose movement vector is zero can be determined to be a pseudo object and removed (S38). As a result, in the light distribution control of the headlight unit 10 (light distribution control that switches from high beam to low beam when an obstacle is detected), which will be described later, pseudo objects that are objects that do not actually exist, such as the reflection of the own vehicle's lights, can be eliminated. It is possible to exclude obstacles from light distribution control. Then, the moving object determination unit 42 determines that an object having the same amount of movement and a vector in the opposite movement direction as the own vehicle is a stationary object, and a different object as a moving object, and sets the movement vector as a moving object attribute in the object information. Add (S39).

In this way, the camera control unit 3 detects the object, determines whether the detected object is a reflective object or a light-emitting object by comparing the brightness images when the headlights are on and when the headlights are off, and identifies the detected object as a reflective or light-emitting object. Attributes are added, relative position attributes including distance information from the own vehicle are added using parallax images, and moving body attributes are added by comparing a plurality of brightness images.

Next, lane recognition will be described. In addition to normal lane detection, when the lane cannot be detected due to blurred lanes, the reflective object information is used to detect lanes using reflectors on guardrails on the roadside or delinators. improve rate.

FIG. 7 shows a flowchart of the lane/road boundary detection unit 43.

First, a brightness image is generated in the brightness image generation unit 32 of the image processing unit 20 (S41). The edge image generation unit 35 generates an edge image from the gradient of the luminance image (S42). The lane/road boundary detection unit 43 detects a lane from the edge image, and calculates a coordinate point according to a certain relative distance from the own vehicle (S43).

In addition to lanes, reflective objects placed at regular intervals on the shoulder of the road are recognized as objects, so information about those objects is used. Among the objects, the lane/road boundary detection unit 43 detects a reflective object with a reflective/luminous object attribute, a stationary object with a moving object attribute, and multiple objects detected at equal intervals in relative position from the own vehicle. It is extracted as a road boundary group (S44). In other words, when reflective objects that are determined to be stationary objects by the lane/road boundary detection unit 43 are placed at equal intervals (at a fixed distance) relative to the host vehicle, they are determined to be a road boundary (group). to decide. Depending on whether a lane is detected on the screen, whether to use lane detection information or road boundary groups is selected as lane information. When a lane is detected on the screen (S45), the lane/road boundary detection unit 43 outputs the coordinate point of the lane detection as lane information (S46). If no lane is detected on the screen (S45), the lane/road boundary detection unit 43 outputs a coordinate point that is a certain distance closer to the own vehicle from the coordinates of the road boundary group as lane information (S47).

Next, a light distribution control method for controlling the high beam and low beam of the headlights using the recognition target information will be explained. Light distribution control is a technology that switches between high beam and low beam depending on the presence or absence of an object, such as high beam when there is no object and low beam when there is an object. A feature of the first embodiment of the present invention is that reflective objects including pseudo objects are excluded from light distribution control targets by using the reflection/emission object attributes of the recognition target (pseudo objects removal section 41).

FIG. 8 shows a flowchart of light distribution control.

If the light distribution control of the headlights is finely switched between high beam and low beam, the driver will feel uncomfortable, so in order to improve the accuracy of obstacle detection, objects are accumulated from multiple images and saved (S51). The presence or absence of the same object in multiple images is checked (S52). If there are identical objects, the reflective/luminous object attributes of the objects are used, and if they are reflective objects, they are treated as if they do not exist (S53). Further, if the object is not a reflective object (S53), it is determined whether the same object is a headlight or taillight of another vehicle detected by the light distribution light detection unit 44 (S54).

When the three conditions (S52, S53, S54) are satisfied, it is determined that there is a valid object that should be set to low beam, and the light distribution instruction unit 71 generates a light distribution instruction to set low beam, and the headlight unit A light distribution instruction is sent to 10 (S55). The light distribution control section 29 of the headlight section 10 controls the headlights to be set to low beam (S56).

If any of the three conditions (S52, S53, S54) is not satisfied, it is determined that there is no valid target to be set to low beam, and control is performed to set high beam. The light distribution instruction unit 71 generates a light distribution instruction to set the high beam, and sends the light distribution instruction to the headlight unit 10 (S57). The light distribution control section 29 of the headlight section 10 controls the headlights to be set to high beam (S58).

The reflective object determination conditions (S53) and target object determination conditions (S54) for light distribution control can be changed and expanded. For example, in this example, the light-emitting object is the headlight or taillight of another vehicle, but if the attribute of a pedestrian or bicycle is to be added, then the condition should be added when a reflective object such as a pedestrian or bicycle is detected. You can also do that.

Next, the vehicle control unit 4 calculates automatic emergency brake control and lane departure control using the object information and lane information obtained from the recognition results of the camera control unit 3.

Figure 9 shows a flowchart of automatic emergency brake control.

The camera control unit 3 sends reflective/luminescent object attributes, relative position attributes, and moving object attributes as object information for recognition (S61). After receiving the object information to be recognized, the automatic emergency brake control section 25 uses the accumulated object information to predict the trajectory of the obstacle in the obstacle trajectory calculation section 60 (S62). The obstacle trajectory calculation unit 60 predicts the trajectory of the light-emitting object using the relative position information of the object information based on the reflective/light-emitting object attribute, and the trajectory of the light-emitting object determined based on the reflective/light-emitting object attribute during the lights-out period. Since it cannot be detected and the relative position is unknown, the trajectory is predicted by predicting the position using linear interpolation or polynomial interpolation from the relative positions of multiple times back (S63). The reflecting object corresponds to both stationary objects and moving objects, so the moving object attribute is not used. However, if you use the moving object attribute, a stationary object with the moving object attribute may be treated as the target object's trajectory, skipping trajectory prediction and treated as stationary, and calculate the relative position of the own vehicle with respect to the own vehicle trajectory. . For example, when the reflecting object is a moving object, the obstacle trajectory calculation unit 60 determines that the moving object is a pedestrian or a bicycle based on the relative speed of the own vehicle, and calculates the future trajectory based on the past trajectory. Predict the trajectory. Then, the risk calculation unit 61 calculates whether the trajectory of the light-emitting object and the reflective object is on the path of the own vehicle, and calculates the risk of collision (S64). The control unit 62 issues a warning or an emergency brake control instruction based on the collision risk calculated by the risk calculation unit 61 (when it is determined that the risk of collision with the host vehicle is high) (S65). Thereafter, the brake unit 13 of the actuator receives the emergency brake control instruction via the CAN 6, and performs brake control of the own vehicle based on the control instruction.

FIG. 10 shows a flowchart of automatic lane departure control.

The camera control unit 3 sends out reflective/luminous object attributes, relative position attributes, moving object attributes, and recognition lane information as recognition object information (S71). The own vehicle trajectory calculation unit 52 predicts the trajectory of the own vehicle from the steering information of the own vehicle (S72). The deviation determination unit 53 inputs the trajectory predicted from the steering information and the recognized lane information, and calculates the risk of deviating from the lane (S73). Based on the lane departure risk determined by the departure determination unit 53, the control unit 51 issues a warning or a steering control instruction to avoid lane departure (to prevent entry into the road boundary) (S74). Thereafter, the steering unit 12 of the actuator receives the steering control instruction via the CAN 6, and performs steering control of the own vehicle based on the control instruction.

[Second example]
Assuming that the duty of turning on and off the headlights is synchronized with the duty of the other vehicle, the headlights of the other vehicle will always be turned off from the perspective of the own vehicle. Therefore, this situation can be avoided by using a multi-duty system that has a plurality of duties for turning on and off the headlights.

The principle of the second embodiment is shown in FIG. It is assumed that there are two cycles, and the ratio of lights off and lights on in the first cycle is 1:9, and the ratio of lights off and lights on in the second cycle is 1:4. That is, the headlights are turned on and off periodically at different duties (1:9 in the first cycle, 1:4 in the second cycle). The imaging timing of the camera is also variable according to the two-cycle on/off duty. By performing phase adjustment, in the first cycle, images are taken twice when the lights are off, and six times when the lights are on, and in the second cycle, images are taken three times when the lights are off, and five times when the lights are on. Repeat this cycle. Two cycles is one example; by using a multi-duty system with three or four cycles and multiple cycles, the possibility of synchronization with other vehicles can be significantly reduced.

The system configuration diagram of the second embodiment may be the same as FIG. 1. The light-off duty calculation section 40 becomes multi-duty, and accordingly, the duty instruction section 70 of the light control section 23 and the light-off duty setting section 28 of the headlight section 10 become light control compatible with multi-duty. In addition, the camera's imaging timing is variable to match the headlights and is compatible with multi-duty. Specifically, the imaging timing calculation section 30 of the image processing section 20, the imaging phase adjustment section 31, and the shutter setting section 27 of the camera imaging section 2 perform variable shutter control corresponding to multi-duty of turning on and off.

The first embodiment and second embodiment of the present invention can also be expanded as follows.

The headlight turn-off duty calculation of the recognition unit 21 can be set variably instead of fixedly in the light-off duty calculation unit 40. In order to variably set the light-off duty, it can be set based on a random number when the ignition is turned on, for example. Further, in order to reduce the frequency of turning off the headlights, it is also possible to turn off the headlights only during a specific period or at a specific location based on the map information of the map section 11.

[Summary of the first and second embodiments]
As described above, the camera control device (camera control section 3) of this embodiment is a camera that controls the headlight section 10 that illuminates the front of the own vehicle and the camera imaging section 2 that takes an image of the front of the own vehicle. In the control device, the camera control device sets a lighting/extinguishing pattern of a predetermined frequency and duty in the headlight section 10 (duty instruction section 70), and turns on/off using the lighting/extinguishing pattern. The headlight unit 10 (light distribution) is controlled (light distribution instruction unit 71), and the camera imaging unit 2 receives a periodic first signal based on a predetermined frequency and duty for the headlight unit 10. An imaging timing pattern different from the lighting/extinguishing pattern is set for an imaging time interval of 1 and a second imaging time interval shorter than the first imaging time interval (imaging timing calculation unit 30), and the imaging timing pattern of the camera imaging unit 2 is set. By adjusting the time phase (imaging phase) of the timing pattern, the imaging time interval when the headlight section 10 is switched between turning off and turning on is reduced (shortened) (imaging phase adjustment section 31).

The camera control device adjusts the imaging timing pattern of the camera imaging unit 2 based on the number of images at the time of lighting and the number of images at the time of turning off of each image captured by the camera imaging unit 2 with the time phase shifted. Adjust the time phase (imaging phase adjustment section 31).

The camera control device measures the brightness of a specific pixel of each image captured by the camera imaging unit 2 while shifting the time phase, determines an image when the light is on and an image when the light is off based on the luminance, and The time phase of the imaging timing pattern of the camera imaging unit 2 is adjusted by checking the number of images when the light is on and the number of images when the light is off periodically at the shutter timing in the time phase (imaging phase adjustment unit 31). .

The camera control device compares a first brightness image and a second brightness image obtained by turning off and turning on the headlight unit 10 captured by the camera imaging unit 2, extracts a reflective object, and generates a plurality of brightness images. The moving vector of the reflecting object in the image is calculated by comparing the moving vectors of the reflecting object, and when the moving vector of the reflecting object is opposite to the moving vector of the own vehicle, the reflecting object is regarded as a stationary object and the moving vector of the reflecting object is zero. When this occurs, the reflective object is determined to be a pseudo object, and in all other cases, the reflective object is determined to be a moving object (luminance image generation section 32, image comparison section 34, pseudo object removal section 41, and moving object determination section 42).

The camera control device variably sets the duty of the headlight section 10 (light-off duty calculation section 40).

The camera control device sets the duty of the headlight unit 10 based on a random number (light-off duty calculation unit 40).

The camera control device periodically turns on and off the headlight section 10 at different duties a plurality of times (light-off duty calculation section 40, duty instruction section 70, light distribution instruction section 71).

The camera control device causes the headlight section 10 to be turned off only during a specific period or at a specific location based on the map information (lights-off duty calculation section 40).

In the light distribution control of the headlight unit 10 that switches from high beam to low beam when an obstacle is detected, the camera control device excludes the pseudo object of the reflective object from the obstacle target of the light distribution control. removal section 41, light distribution instruction section 71).

Further, the vehicle control device (vehicle control unit 4) of this embodiment is a vehicle control device that controls the own vehicle based on information output from the camera control device, and the vehicle control device controls the own vehicle based on the information output from the camera control device. If the reflective object determined to be a stationary object is located at a certain distance, it is determined to be a road boundary, and an alarm is issued or the steering of the own vehicle is controlled to prevent intrusion into the road boundary (lane departure). control unit 24).

Further, the vehicle control device (vehicle control unit 4) of this embodiment is a vehicle control device that controls the own vehicle based on information output from the camera control device, and the vehicle control device controls the own vehicle based on the information output from the camera control device. If the reflective object is the moving object, it is determined that the moving object is a pedestrian or a bicycle based on the relative speed of the vehicle, the future trajectory is predicted from the past trajectory, and the vehicle The risk of a collision with the own vehicle is calculated, and when it is determined that the risk of a collision with the own vehicle is high, a warning or automatic emergency brake control is performed (automatic emergency brake control unit 25).

As described above, according to this embodiment, reflective objects and light-emitting objects can be detected by periodically turning off the headlights for short periods of time and capturing images at short timings at the boundary between turning off and turning on the headlights. Among reflective objects, by correctly removing pseudo-objects that appear as objects on the road due to reflections from headlights, etc., it is possible to avoid incorrect headlight light distribution control or automatic emergency braking caused by pseudo-objects. Furthermore, various advanced driving support systems (such as those that use reflective objects such as curbs and guardrails as road boundaries and use them as substitutes when lane detection is not possible, improving the performance of automatic departure avoidance systems) are being developed. Advanced Driver Assistance Systems (ADAS) can be realized.

Note that the present invention is not limited to the embodiments described above, and includes various modifications. For example, the embodiments described above are described in detail to explain the present invention in an easy-to-understand manner, and the present invention is not necessarily limited to having all the configurations described.

Further, each of the above-mentioned configurations, functions, processing units, processing means, etc. may be partially or entirely realized in hardware by, for example, designing an integrated circuit. Further, each of the above-mentioned configurations, functions, etc. may be realized by software by a processor interpreting and executing a program for realizing each function. Information such as programs, tables, files, etc. that realize each function can be stored in a memory, a storage device such as a hard disk, an SSD (Solid State Drive), or a recording medium such as an IC card, SD card, or DVD.

In addition, control lines and information lines are shown that are considered necessary for explanation, and not all control lines and information lines are necessarily shown in the product. In reality, almost all components may be considered to be interconnected.

1...Vehicle 2...Camera imaging section 3...Camera control section (camera control device)
4...Vehicle control unit (vehicle control device)
5...Luminometer 6...CAN
10... Headlight section 11... Map section 12... Steering section 13... Brake section 20... Image processing section 21...

Recognition section

22, 26... CAN-IF section 23. ...Light control section 24...Lane departure control section 25...Automatic emergency brake control section 27...Shutter setting section 28...Lights off duty setting section 29...Light distribution control section 30...Image capture Timing calculation unit 31...imaging phase adjustment unit 32...luminance image generation unit 33...parallax image generation unit 34...image comparison unit 35...edge image generation unit 40...light-off duty calculation unit 41... Pseudo object removal unit 42... Moving object determination unit 43... Lane/road boundary detection unit 44... Light distribution

light detection unit

51, 62... Control unit 52... Self-vehicle Trajectory calculation unit 53... Deviation determination unit 60... Obstacle trajectory calculation unit 61... Risk calculation unit 70... Duty instruction unit 71... Light distribution instruction unit 100... Oncoming vehicle at time t1 Headlights 101...Reflection on a reflective object on the curb at time t1 102...Reflection on a reflective object on a guardrail at time t1 103...Reflection on the own vehicle's headlight at time t1 (pseudo object)
104... Headlight of an oncoming vehicle at time t2 111... Reflective object of a bicycle in the roadway at time t1... Reflective object of a stationary object outside the roadway at time t1 113... Inside the roadway at time t1 Pseudo object 114 due to the reflection of the headlights...Reflection object 115 of a bicycle on the roadway at time t2...Reflection object 116 of a stationary object outside the roadway at time t2...Reflection of the headlights on the roadway at time t2 Pseudo object due to reflection 117...Movement vector of the reflective object of the bicycle on the roadway at time t2-t1...Movement vector of the reflective object of a stationary object outside the roadway at time t2-t1...Time t2- Movement vector of the pseudo object due to reflection of headlights in the roadway at t1

Claims

A camera control device that controls a headlight unit that illuminates the front of the host vehicle, and a camera imaging unit that captures an image of the front of the host vehicle,
The camera control device includes:
Setting a lighting/extinguishing pattern with a predetermined frequency and duty in the headlight section, and controlling the headlight section to turn on/off using the lighting/extinguishing pattern;
The camera imaging unit has a regular first imaging time interval and a second imaging time interval shorter than the first imaging time interval, based on a predetermined frequency and duty for the headlight unit. By setting an imaging timing pattern different from a lighting-on/off pattern and adjusting the time phase of the imaging timing pattern of the camera imaging section, an imaging time interval when switching between turning off and turning on the headlight section is reduced. , a camera control device.
The camera control device adjusts the time phase of the imaging timing pattern of the camera imaging unit based on the number of images at the time of lighting and the number of images at the time of turning off of each image captured by the camera imaging unit with the time phase shifted. The camera control device according to claim 1, wherein the camera control device adjusts.
The camera control device measures the brightness of a specific pixel of each image captured by the camera imaging unit while shifting the time phase, determines an image when the light is on and an image when the light is off based on the luminance, and calculates the time 10. The time phase of the imaging timing pattern of the camera imaging unit is adjusted by checking the number of images when the light is on and the number of images when the light is off periodically with shutter timing in the phase. 1. The camera control device according to 1.
The camera control device compares a first brightness image and a second brightness image obtained by turning off and turning on the headlight unit captured by the camera imaging unit, extracts a reflective object, and compares the plurality of brightness images. to calculate the movement vector of the reflecting object in the image, and when the movement vector of the reflecting object is opposite to the movement vector of the host vehicle, the reflecting object is a stationary object, and when the moving vector of the reflecting object is zero, the reflecting object is a stationary object. 2. The camera control device according to claim 1, wherein the reflective object is determined to be a pseudo object when the reflective object is determined to be a pseudo object, and the reflective object is determined to be a moving object at other times.
The camera control device according to claim 1, wherein the camera control device variably sets the duty of the headlight section.
The camera control device according to claim 5, wherein the camera control device sets the duty of the headlight section based on a random number.
2. The camera control device according to claim 1, wherein the camera control device periodically turns on and off the headlight unit a plurality of times with different duties.
The camera control device according to claim 1, wherein the camera control device turns off the headlight section only during a specific period or at a specific location based on map information.
The camera control device is characterized in that, in light distribution control of the headlight section that switches from high beam to low beam when an obstacle is detected, the pseudo object of the reflective object is excluded from the obstacle target of the light distribution control. The camera control device according to claim 4.
A vehicle control device that controls the host vehicle based on information output from the camera control device according to claim 4,
If the reflective object determined to be a stationary object is located at a certain distance, the vehicle control device determines that the reflective object is a road boundary, and issues an alarm to prevent the own vehicle from invading the road boundary. A vehicle control device that controls steering.
A vehicle control device that controls the host vehicle based on information output from the camera control device according to claim 4,
When the reflective object is the moving object, the vehicle control device determines that the moving object is a pedestrian or a bicycle based on a comparison with the relative speed of the own vehicle, and determines the future trajectory from the past trajectory. The present invention is characterized by predicting the possibility of a collision with the own vehicle, calculating the risk of a possibility of a collision with the own vehicle, and performing a warning or automatic emergency braking control when it is determined that the risk of a possibility of a collision with the own vehicle is high. , vehicle control equipment.