JP6967866B2 - Image display device - Google Patents

Description

The present invention relates to an image display device provided in an automobile that changes lanes by automatic steering, and particularly to a device that can easily confirm a lateral situation during standby for automatic steering.

An image that displays an image taken by a camera provided on the side or rear of a vehicle body in place of or in combination with existing mirrors as a vehicle rear side monitoring means in a vehicle such as an automobile. It has been proposed to use a display device (camera monitoring system).
As a conventional technique relating to such an image processing device, for example, in Patent Document 1, an image is displayed in an inner region and an outer region of a steering wheel rim portion in a front view of a driver, and each captured image straddles the rim portion. It describes what is configured to move.

Further, in recent years, in vehicles such as automobiles, driving support devices and automatic driving devices that recognize the environment around the vehicle and automatically perform steering operations such as lane keeping and lane change are becoming widespread.
Patent Document 2 describes automatic driving control when it is determined in the automatic driving support system that automatic driving control cannot be implemented at present or in the future while driving in an automatic driving section where automatic driving control of a vehicle is permitted. It is described that the reason for non-implementation, which is the reason why the vehicle cannot be implemented, is acquired, the automatic operation control cannot be implemented, and the acquired reason for non-implementation is also guided by a character string or voice.

Patent No. 5903319 Japanese Unexamined Patent Publication No. 2016-28927

When changing lanes from the lane in which the own vehicle is traveling to the adjacent adjacent lane by using a driving support device or an automatic driving device having an automatic steering function, for example, an obstacle such as another vehicle is approaching from the rear side. For this reason, it may be necessary to wait for the steering operation to change lanes.
In such a case, for example, as described in Patent Document 2, it is conceivable to guide the reason for waiting for the steering operation by characters, voice, etc., but in this case, in order for the user to visually recognize the obstacle later. It is necessary to look back to the side and visually confirm it, and it is required that the user (occupant) can more intuitively and easily grasp the reason why the steering standby state is set.
In view of the above-mentioned problems, an object of the present invention is to provide an image display device capable of easily confirming a lateral situation during standby for automatic steering.

The present invention solves the above-mentioned problems by the following solution means.
The invention according to claim 1 is an image display provided on a vehicle including an environment recognition means for recognizing the environment around the own vehicle and a driving control means for automatically performing a steering operation based on the recognition result of the environment recognition means. When the device is in a steering standby state in which the image display means arranged in front of the occupant and the operation control means wait for the steering operation according to the recognition result of the environment recognition means, the device is in a steering standby state. It is provided with a display control means for displaying a side situation image displaying the side situation in the planned steering direction on the image display means, and the position of the side situation image in the front-rear direction of the own vehicle laps with the own vehicle. with it included within the display range at least part of the neighbor lane that put the range, the side view of another vehicle running parallel to the side of the vehicle the environment recognizing means recognizes the own vehicle front and rear It is an image display device characterized by displaying together with a line indicating the position of an end portion.
According to this, when the steering standby state occurs, the user can immediately confirm the obstacle causing the steering standby without looking back to the rear side by displaying the lateral situation in the planned steering direction. It is possible to intuitively and easily grasp the reason why the steering standby is performed and the prospect that the steering standby will end.
In addition, by displaying the situation in the area where there is a risk of collision with an obstacle when the own vehicle moves laterally, it is possible to appropriately grasp the cause of the steering standby.
Further, when the steering standby is performed due to the risk of contact with another vehicle, the reason for the steering standby can be appropriately recognized by displaying the other vehicle.

The invention according to claim 2, wherein the steering operation is for the vehicle performs a lane change from the lane currently driving to the adjacent lane, the lateral status image, the adjacent lane relative to the vehicle The image display device according to claim 1, wherein the situation on the side of the side is displayed.
According to this, when the driving control device automatically changes lanes, the above-mentioned effect can be surely obtained.

Invention, said lateral status image may be displayed in response to said termination of a steering standby state with the display in response to the start of the steering standby state of the operation control means is started is terminated according to claim 3 The image display device according to claim 1 or 2, wherein the image display device is characterized by the above-mentioned.
According to the fourth aspect of the present invention, the display of the side situation image is started according to the start of the steering standby state of the operation control means, and the display is terminated according to the end of the steering operation. The image display device according to claim 1 or 2, which is a feature.
According to each of these inventions, after displaying the side situation image for a sufficient period of time to inform the user of the cause of the steering standby, it is possible to return to the original display at an appropriate timing.

As described above, according to the present invention, it is possible to provide an image display device that can easily confirm the lateral situation at the time of automatic steering standby.

It is a block diagram which shows the structure of embodiment of the image display apparatus to which this invention is applied. It is a schematic perspective view which shows the arrangement of the rear side camera and the like in the image display device of embodiment. It is a figure which shows an example of the user's field of view in the vehicle which has the image display device of embodiment. It is a block diagram which shows the structure of the driving control device provided in the vehicle which has the image display device of embodiment. It is a flowchart which shows the operation of the image display device of an embodiment. It is a figure which shows an example of the display image in a normal time in the image display apparatus of embodiment. It is a figure which shows an example of the display image at the time of lane change standby in the image display device of embodiment. It is a figure which shows the other example of the display image at the time of lane change standby in the image display device of embodiment.

Hereinafter, embodiments of an image display device to which the present invention is applied will be described.
The image display device of the embodiment is a camera monitoring system provided in an automobile such as a passenger car, and presents an image or the like captured by a camera on the rear side of the vehicle body to a user such as a driver.

FIG. 1 is a block diagram showing a configuration of an image display device according to an embodiment.
The image display device 1 includes a rear side image processing unit 10, a rear camera 11, a left rear side camera 12, a right rear side camera 13, a stereo camera processing unit 20, a left front camera 21, a right front camera 22, and environment recognition. It includes a unit 30, a left rear side radar 31, a right rear side radar 32, a driver monitoring unit 40, a driver monitoring camera 41, a display image generation unit 50, a display control unit 60, an LCD 70, and the like.

The rear side image processing unit 10 is a digital signal processing device that processes the output signals of the rear camera 11, the left rear side camera 12, and the right rear side camera 13 and outputs them as image data.
The video signal output by the rear side image processing unit 10 is transmitted to the environment recognition unit 30 and the display image generation unit 50.
The rear camera 11, the left rear side camera 12, and the right rear side camera 13 capture a moving image of the situation around the vehicle and output a video signal in real time, and are a solid-state image sensor such as CMOS, a drive device thereof, and a lens. Etc. are provided.

FIG. 2 is a schematic perspective view showing the arrangement of a rear side camera or the like in the image display device of the embodiment.
The rear camera 11 is arranged at the central portion in the vehicle width direction at the rear end portion of the vehicle body B with the optical axis direction facing the rear side of the vehicle.
The left rear side camera 12 is arranged on the left side surface portion of the vehicle body B with the optical axis direction facing the rear side and the vehicle width direction outward.
The left rear side camera 12 is arranged at a position adjacent to the side mirror 130, for example.
The field of view (capable of imaging) F12 of the left rear side camera 12 partially wraps with the field of view F11 of the rear camera 11, and the entire range from the rear of the own vehicle to the left side is substantially covered by at least one of the cameras. The image is taken so that there is no blind spot.
The right rear side camera 13 (not shown in FIG. 2) is arranged substantially symmetrically with the left rear side camera 12.

The stereo camera processing unit 20 applies known stereo camera processing to the images captured by the left front camera 21 and the right front camera 22, respectively, and detects an object in front of the own vehicle and the distance from the own vehicle.
The output of the stereo camera processing unit 20 is transmitted to the environment recognition unit 30.
The left front camera 21 and the right front camera 22 capture a moving image of the situation in front of the vehicle and output a video signal in real time, and include a solid-state image sensor such as CMOS, a drive device thereof, a lens, and the like.
The left front camera 21 and the right front camera 22 are arranged, for example, near the upper end of the front window glass in the vehicle interior, apart from each other in the left-right direction.

The environment recognition unit 30 recognizes the environment around the own vehicle based on the information provided from the rear side image processing unit 10, the stereo camera processing unit 20, the left rear side radar 31, the right rear side radar 32, and the like. It is something to do.
The environment recognition unit 30 functions as another vehicle detection means according to the present invention in cooperation with the left rear side radar 31 and the right rear side radar 32.
Here, as the recognized environment, the lane shape of the own vehicle traveling lane, the lane shape of the adjacent lane adjacent to the own vehicle traveling lane, other vehicles traveling around the own vehicle, pedestrians, and buildings. Includes obstacles such as (eg, buildings, guardrails, signs, traffic lights, bridges, tunnels, central separation zones, roadside zones, etc.).

The left rear side radar 31 and the right rear side radar 32 detect an object such as another vehicle existing on the rear side of the own vehicle.
As the left rear side radar 31 and the right rear side radar 32, for example, a millimeter wave radar can be used.
The left rear side radar 31 and the right rear side radar 32 can detect the relative position of the other vehicle with respect to the own vehicle when another vehicle exists, and the other vehicle is based on the transition of this relative position. It is possible to calculate the relative speed of the vehicle.
The left rear side radar 31 and the right rear side radar 32 are arranged on the left and right sides (for example, inside the rear bumper) in the rear part of the vehicle body.

The driver monitoring unit 40 detects the posture, behavior, line-of-sight direction, etc. of the user (driver during manual driving) based on the image captured by the driver monitoring camera 41.
The driver monitoring camera 41 is provided on the front side of the driver in the vehicle interior.

The display image generation unit 50 generates a display image to be displayed on the LCD 70 based on the inputs from the rear side image processing unit 10, the environment recognition unit 30, and the driver monitoring unit 40.
The function of the display image generation unit 50 and an example of the generated display image will be described in detail later.
The displayed image may be a live-action image obtained by performing predetermined image processing on the image captured by the rear camera 11, the left rear side camera 12, and the right rear side camera 13, and corresponds to the live-action image. It may be computer graphics or the like for displaying information. Further, the live-action image and the computer graphics may be superimposed and used in combination.

The display control unit 60 is a display drive device that displays a display image generated by the display image generation unit 50 on the LCD 70.
The LCD 70 is a display means provided on the front side of the user (passenger / driver during manual driving) to display a display image to the user.
The arrangement of the LCD 70 in the vehicle interior will be described in detail later.

FIG. 3 is a diagram showing an example of a user's field of view in a vehicle having the image display device of the embodiment.
As shown in FIG. 3, the vehicle has a windshield 110, a front door glass 120, a side mirror 130, an instrument panel 140, a steering wheel 150, an A pillar 160, a roof 170, a rearview mirror 180 and the like.

The windshield 110 is arranged on the front side of the driver.
The windshield 110 is a quadric curved glass that is formed in a substantially horizontally long rectangular shape and is curved in a direction in which the front is convex.
The windshield 110 is arranged so as to be tilted backward so that the upper end portion is on the rear side of the vehicle with respect to the lower end portion.

The front door glass 120 is an upper portion of the left and right front doors used for passengers to get on and off, and is provided on the side of the driver.
The front door glass 120 has an elevating main body portion 121 and a fixed triangular window portion 122 provided at the front portion of the main body portion 121.

The side mirror 130 allows the driver to confirm the left-right rear view.
The side mirrors 130 project outward in the vehicle width direction from the outer panels of the left and right front doors.
In the user's field of view, the side mirror 130 is visible, for example, in the vicinity of the front end portion of the main body portion 121 of the front door glass 120.

The instrument panel 140 is an interior member provided below the windshield 110 in the vehicle interior.
The instrument panel 140 also functions as a housing for accommodating various instruments, display devices, switches, air conditioners, passenger seat airbag devices, knee protection airbag devices, and the like.
The instrument panel 140 is provided with a combination meter 141, a multifunction display 142, a navigation device display 143, and the like.

The combination meter 141 is provided in front of the driver's seat, and is a unitization of various instruments such as a speedometer, an engine tachometer, and a range finder.
The combination meter 141 has a built-in LCD 70.
The multifunction display 142 is an image display means such as an LCD provided at the upper portion of the instrument panel 140 in the central portion in the vehicle width direction.
The navigation device display 143 is provided at the lower portion of the instrument panel 140 in the central portion in the vehicle width direction.

The steering wheel 150 is an annular operating member for inputting a steering operation during manual operation by the driver.
The steering wheel 150 is provided substantially directly in front of the driver.
The combination meter 141 is visible from the inner diameter side in the upper half of the steering wheel 150 in the user's field of view.

The A-pillar 160 is a columnar vehicle body structural member arranged along the side end portion of the windshield 110 and the front end portion of the windshield 120.
The surface of the A-pillar 160 on the vehicle interior side is covered with a resin pillar trim.

The roof 170 is formed so as to extend rearward from the upper end portion of the windshield 110.
The surface of the roof 170 on the vehicle interior side is covered with a resin roof trim.
A stereo camera accommodating portion 171 for accommodating the left front camera 21 and the right front camera 22 is provided at the front end portion of the roof 170 in the central portion in the vehicle width direction.

The rearview mirror 180 is a rear-view mirror provided in the vehicle interior.
The rearview mirror 180 is provided near the upper end portion of the windshield 110 in the center portion in the vehicle width direction via a stay (not shown).

The vehicle is equipped with a driving control device described below.
This operation control device constitutes an automatic operation system having an automatic operation function that automatically performs operation operations such as steering and acceleration / deceleration according to the recognition result by the environment recognition unit 30.
FIG. 4 is a block diagram showing a configuration of a driving control device provided in a vehicle having the image display device of the embodiment.
As shown in FIG. 4, the operation control device includes an automatic operation control unit 200, an engine control unit 210, a transmission control unit 220, a behavior control unit 230, an electric power steering (EPS) control unit 240, and the like.
Each of the above-mentioned units is configured as a unit having, for example, an information processing means such as a CPU, a storage means such as a RAM or a ROM, an input / output interface, and a bus connecting them. Each of these units can communicate with each other via an in-vehicle LAN system such as a CAN communication system.

When the automatic driving mode is selected by the user, the automatic driving control unit 200 outputs a control command to the engine control unit 210, the transmission control unit 220, the behavior control unit 230, the EPS control unit 240, and the like to automatically drive the vehicle. It executes automatic driving control to drive the vehicle.

When the automatic driving mode is selected, the automatic driving control unit 200 advances the own vehicle in response to information on the situation around the own vehicle provided by the environment recognition unit 30 and a command from a driver (not shown). Set the target driving trajectory to be.
The automatic driving control unit 200 automatically accelerates (starts), decelerates (stops), switches forward / backward, and steers the vehicle so that the actual traveling locus of the own vehicle substantially matches the target traveling locus. , Executes automatic driving that automatically drives the vehicle to a preset destination.
In addition, the automatic operation mode is canceled according to a predetermined release operation from the user when the user desires manual operation or when it is difficult to continue the automatic operation, and the manual operation is performed by the driver. It is possible to return to the operation mode.
Further, the automatic operation control unit 200 is capable of communicating with the display image generation unit 50, and gives an instruction to the display image generation unit 50 to switch the mode of the image to be generated. This point will be described in detail later.

An input / output device 201 is connected to the automatic operation control unit 200.
The input / output device 201 outputs information such as alarms and various messages to the user from the automatic operation control unit 200, and also receives inputs of various operations from the user.
The input / output device 201 includes, for example, an image display device such as an LCD, an audio output device such as a speaker, an operation input device such as a touch panel, and the like.
The input / output device 201 can also use, for example, an LCD 70, a multifunction display 142, or the like as a touch panel.

The engine control unit 210 comprehensively controls the engine, which is a power source for traveling the vehicle, and its accessories.
As the engine, for example, a 4-stroke gasoline engine is used.
The engine control unit (ECU) 210 can control the output torque of the engine by controlling the throttle valve opening degree of the engine, the fuel injection amount and the injection timing, the ignition timing, and the like.
When the vehicle is driven according to the driver's driving operation, the engine control unit 210 outputs the engine so that the actual torque of the engine approaches the driver's required torque set based on the operation amount of the accelerator pedal and the like. To control.
Further, when the vehicle automatically drives, the engine control unit 210 controls the output of the engine in response to a command from the automatic driving control unit 200.

The transmission control unit (TCU) 220 shifts the rotational output of the engine and comprehensively controls a transmission and accessories (not shown) for switching between forward and reverse of the vehicle.
When the vehicle automatically drives, the transmission control unit 220 switches the range such as forward / backward movement and sets the gear ratio in response to a command from the automatic driving control unit 200.
As the transmission, for example, a chain type, a belt type, a toroidal type or the like CVT, or various automatic transmissions such as a step AT having a plurality of planetary gear sets, a dual clutch transmission, an AMT and the like can be used.
The transmission is configured to have a transmission mechanism such as a variator, a start device such as a torque converter, a dry clutch, and a wet clutch, and a forward / backward switching mechanism for switching between a forward travel range and a reverse travel range. There is.

A forward / backward switching actuator 221, a range detection sensor 222, and the like are connected to the transmission control unit 220.
The forward / backward switching actuator 221 drives a forward / backward switching valve that switches an oil passage that supplies oil to the forward / backward switching mechanism, and switches the forward / backward movement of the vehicle.
The forward / backward switching actuator 221 is, for example, an electric actuator such as a solenoid.
The range detection sensor 222 is a sensor (switch) that determines whether the range currently selected in the transmission is for forward movement or backward movement.

The behavior control unit 230 controls behavior that suppresses vehicle behavior such as understeer and oversteer by individually controlling the wheel cylinder hydraulic pressure of the hydraulic service brakes provided on the left, right, front and rear wheels, and wheel lock during braking. It performs anti-lock brake control to recover.
A hydraulic control unit (HCU) 231, a vehicle speed sensor 232, and the like are connected to the behavior control unit 230.

The HCU 231 includes an electric pump that pressurizes the brake fluid, which is the working fluid of the hydraulic service brake, and a valve that individually adjusts the hydraulic pressure supplied to the wheel cylinders of each wheel.
When the vehicle automatically drives, the HCU 231 generates a braking force in the wheel cylinders of each wheel in response to a braking command from the automatic driving control unit 200.
The vehicle speed sensor 232 is provided on the hub portion of each wheel and generates a vehicle speed pulse signal having a frequency proportional to the rotation speed of the wheels.
By detecting the frequency of the vehicle speed pulse signal and performing predetermined arithmetic processing, it is possible to calculate the traveling speed (vehicle speed) of the vehicle.

The electric power steering (EPS) control unit 240 comprehensively controls the electric power steering device that assists the steering operation by the driver by the electric motor and its accessories.
A motor 241 and a steering angle sensor 242 are connected to the EPS control unit 240.

The motor 241 is an electric actuator that applies an assist force to the steering system of the vehicle to assist the steering operation by the driver, or changes the steering angle during automatic driving.
When the vehicle automatically drives, the motor 241 applies torque to the steering system so that the steering angle of the steering system approaches a predetermined target steering angle in response to a steering command from the automatic driving control unit 200. Have them steer.
The steering angle sensor 242 detects the current steering angle in the steering system of the vehicle.
The steering angle sensor 242 includes, for example, a position encoder that detects an angular position of the steering shaft.

The image display device 1 of the embodiment is a steering standby in which the automatic driving control unit 200 waits for a steering operation due to, for example, the approach of an obstacle such as another vehicle after setting a target traveling locus including a lane change. It is characterized in that an image (side situation image I2) of the rear side of the own vehicle is displayed so that the user can confirm the obstacle that caused the situation when the state is reached.
Hereinafter, a more detailed description will be given.
FIG. 5 is a flowchart showing the operation of the image display device of the embodiment, and shows a state when the automatic driving control unit 200 sets a target traveling locus including a lane change accompanied by a steering operation.
Hereinafter, each step will be described step by step.

<Step S01: Target travel trajectory setting>
The automatic driving control unit 200 sets a target traveling locus of the own vehicle based on the environment recognition result of the environment recognition unit 30, input from the user, and the like.
Then, the process proceeds to step S02.

<Step S02: Judgment of steering schedule>
The automatic driving control unit 200 determines whether or not it is necessary to perform a steering operation of a predetermined steering angle or more when the own vehicle is driven along the target traveling locus set in step S01.
If a steering operation of a predetermined steering angle or more is required, the process proceeds to step S03 with a steering schedule, and in other cases (when the vehicle goes straight substantially), a series of processes is terminated (returned) with no steering schedule.

<Step S03: Steering direction environment recognition>
The environment recognition unit 30 newly recognizes the environment in the direction in which steering is scheduled, and detects an obstacle newly recognized after setting the target travel locus or an approaching obstacle.
For example, when the target driving locus is accompanied by a lane change, the following vehicles traveling in the changed lane are detected by the left and right rear side cameras 12, 13 and the rear side radars 31, 32 and the like. ..
After that, the process proceeds to step S04.

<Step S04: Steering standby determination>
Based on the recognition result of the environment recognition unit 30 in step S03, the automatic operation control unit 200 determines whether or not steering standby is required to temporarily wait for the steering operation.
For example, if the target driving locus includes a lane change and the following vehicle traveling in the changed lane is approaching at a relative speed equal to or higher than a predetermined value, it is determined that the contact risk is high, and the automatic driving control unit 200 determines. , Determines to perform steering standby (lane change standby) until the following vehicle overtakes the own vehicle and a space on the side of the own vehicle becomes available.
If steering standby is required, the process proceeds to step S05, and if steering standby is not required (when steering operation is possible according to the initial target travel locus), the process proceeds to step S09.

<Step S05: Start displaying side view image>
The automatic driving control unit 200 instructs the display image generation unit 50 to display the side situation image I2 (see FIG. 7), which is an image showing the situation on the rear side of the own vehicle.
The display image generation unit 50 generates image data related to the side situation image I2, provides it to the display control unit 60, and displays it on the LCD 70.
In the state where the side situation image I2 is not displayed, the normal display image I1 (see FIG. 6) is displayed on the LCD 70.
The normal display image I1 and the lateral situation image I2 will be described in detail later.
Then, the process proceeds to step S06.

<Step S06: Judgment of end of steering standby>
The automatic operation control unit 200 determines whether or not the steering standby state has ended.
The determination of the end of the steering standby state is determined based on the situation on the side of the own vehicle sequentially recognized by the environment recognition unit 30.
For example, when another vehicle of the adjacent vehicle completes the overtaking of the own vehicle and the space on the side of the own vehicle becomes empty, the end of the steering standby is determined.
When the steering standby state ends, the process proceeds to step S07, and when the steering standby state does not end, step S06 is repeated at predetermined time intervals.

<Step S07: Steering operation start>
The automatic operation control unit 200 issues a command to the EPS control unit 240 to start giving a steering angle (steering operation) to the steering system by the motor 241.
Then, the process proceeds to step S08.

<Step S08: Side situation image display end>
The automatic operation control unit 200 ends the generation of the side situation image I2 in the display image generation unit 50, and returns the LCD 70 to the state of displaying the normal display image I1.
After that, a series of processing is completed (returned).

<Step S09: Steering operation start>
The automatic operation control unit 200 issues a command to the EPS control unit 240 to start giving a steering angle (steering operation) to the steering system by the motor 241.
After that, a series of processes is terminated.

Next, the display image of the LCD 70 in this embodiment will be described.
The display image described below is generated by the display image generation unit 50.
FIG. 6 is a diagram showing an example of a display image in a normal time in the image display device of the embodiment.
FIG. 7 is a diagram showing an example of a display image during lane change standby in the image display device of the embodiment.

In the normal state (other than the steering standby state), as shown in FIG. 6, the normal time display image I1 which is a bird's-eye view (bird's-eye view) seen from the rear side and the diagonally upper side of the own vehicle is displayed.
In the normal display image I1, for example, the own vehicle 310, the own vehicle traveling lane 320, the adjacent lane 330, the preceding vehicle 340, the target traveling locus display 350, and the like are displayed.
Such a bird's-eye view is obtained by combining the images captured by each camera after performing predetermined image processing, or by using the display image generation unit 50 in the display image generation unit 50 based on the environment recognition result of the environment recognition unit 30. Generated by rendering as graphics.
Further, instead of such a bird's-eye view (bird's-eye view), for example, a plan view showing a state in which the own vehicle and its surroundings including substantially all directions are viewed from directly above may be used.

FIG. 6 shows a state in which the own vehicle 310 is traveling on a high-standard road (expressway or the like) having two lanes on each side having a own vehicle traveling lane 320 and adjacent lanes 330 running in parallel in the same direction.
The own vehicle 310 is traveling in the own vehicle traveling lane 320.
The adjacent lane 330 is adjacent to the right side of the own vehicle traveling lane 320 and is provided in parallel.
In the own vehicle traveling lane 320, the preceding vehicle 340 is traveling in the same direction as the own vehicle 310 in front of the own vehicle 310.

When the automatic driving control unit 200 sets a target driving locus to overtake the preceding vehicle 340 in order to avoid the preceding vehicle 340, the own vehicle 310 changes lanes from the own vehicle traveling lane 320 to the adjacent lane 330 in the normal display image I1. The target travel locus display 350 indicating what to do is displayed.
The user recognizes that the automatic driving control unit 200 is about to execute a lane change based on the appearance of the target travel locus display 350 and the start of blinking of the turn signal (turn signal) lamp (not shown).

In FIG. 7, a side situation image I2 showing the situation on the right rear side of the own vehicle 310 captured by the right rear side camera 13 is displayed.
The side situation image I2 includes at least a part of the adjacent lane 330 in the range where the position of the own vehicle 310 in the front-rear direction wraps with the own vehicle within the display range, and can cover a field of view substantially equivalent to that of the side mirror 130. It has become.
Such a lateral situation image I2 is a real image obtained by subjecting an image captured by the right rear lateral camera 13 to image processing such as adjustment of brightness, brightness, contrast, color tone, etc., and correction of distortion due to lens aberration. Images can be used.
Further, the image captured by the right rear side camera 13 or the computer graphics rendered based on the output of the right rear side radar 32 may be used.
In addition, in the embodiment, the side situation image is a left-right reverse image because it does not cause a sense of discomfort when visually recognizing the rear side using a general side mirror 130.

In FIG. 7, in addition to the own vehicle 310, the own vehicle traveling lane 320, and the adjacent lane 330, the following vehicle 360 traveling in the adjacent lane 330 is displayed.
The following vehicle 360 is traveling in the adjacent lane 330 at a higher speed than the own vehicle 310 in the same direction.

FIG. 8 is a diagram showing another example of a display image during lane change standby in the image display device of the embodiment.
In FIG. 8, an ultra-wide-angle camera capable of capturing an image of the side of the own vehicle as the right rear side camera 13 and arranged with the optical axis direction substantially along the vehicle width direction is used, and the side of the own vehicle is used. A side situation image I3 showing the situation is displayed.
In FIG. 8, the following vehicle 360 is substantially displayed as a side view.
The side display image I3 displays a front end line Lf indicating the front-rear position of the front end of the own vehicle and a rear end line Lr indicating the front-rear position of the rear end of the own vehicle, and the user can line with other vehicles and the like. Based on the positional relationship with Lf and Lr, it is possible to determine whether or not the parallel running state in overtaking or the like has ended.
For the side display images I2 and I3, for example, the user can select and set which image is displayed in advance.
Further, the side display image I2 may be sequentially switched to the side display image I3 according to the approach of the following vehicle 360 to the own vehicle 310.

When the automatic driving control unit 200 is determined to be concerned about interference with the following vehicle 360 (rear-end collision, etc.) when the own vehicle 310 faithfully travels according to the initially set target traveling trajectory, the following vehicle Until the overtaking of the own vehicle 310 by 360 is completed, the driving of the own vehicle lane 320 is continued in the steering standby state.
The user intuitively understands that the cause of the steering standby state is the following vehicle 360 based on the side situation images I2 and I3 displayed by the LCD 70 arranged in front of the user without looking back to the rear side. It can be easily and targetedly recognized, and it is possible to predict how long the steering standby state will be continued in the future by looking at the position and speed of the following vehicle 360 (state of approach to the own vehicle).

According to the embodiment described above, the following effects can be obtained.
(1) When the steering standby state occurs, by displaying the lateral situation in the planned steering direction as the lateral situation image I2, the user does not look back to the rear side and causes the steering standby. Obstacles such as the vehicle 360 can be immediately confirmed, and the reason why the steering standby is performed and the prospect of the end of the steering standby can be intuitively and easily grasped.
(2) The steering operation is related to the lane change, and when the side situation image I2 displays the situation of the adjacent lane 330 of the lane change destination and automatically changes the lane by the automatic driving control, the above-mentioned The effect can be surely obtained.
(3) The side situation image I2 includes at least a part of the adjacent lane 330 in the range where the position of the own vehicle 310 in the front-rear direction wraps with the own vehicle within the display range, so that the side situation image I2 includes the lateral movement of the own vehicle 310. The situation in the area where there is a risk of collision with the following vehicle 360 or the like can be displayed, and the cause of the steering standby can be appropriately grasped.
(4) By including the range corresponding to the field of view of the side mirror 130 in the display range, the side situation image I2 allows the user to intuitively grasp the cause of the steering standby with a familiar field of view.
(5) When the steering standby is performed due to the risk of contact with the following vehicle 360, the reason for the steering standby can be appropriately recognized by displaying the following vehicle 360 on the side situation image I2.
(6) The side situation image I2 causes the user to wait for steering by starting the display according to the start of the steering standby state and ending the display according to the end of the steering standby state. After displaying the side situation image I2 for a sufficient period of time to inform, it is possible to return to the normal display image I1 at an appropriate timing.

(Modification example)
The present invention is not limited to the embodiments described above, and various modifications and changes can be made, and these are also within the technical scope of the present invention.
(1) The configuration of the image display device and the vehicle on which the image display device is mounted is not limited to the above-described embodiment, and can be appropriately changed.
For example, the quantity and arrangement of cameras and radars can be changed as appropriate. In the embodiment, one rear camera and one left and right rear side cameras are provided, but a plurality of rear cameras may be provided. Further, for example, a single wide-angle camera may be used, and the rear camera and the rear side camera may be used in combination.
(2) In the embodiment, the side situation image is displayed on the LCD 70 provided in front of the user (driver during manual operation), but the present invention is not limited to this, and the display provided in the center of the instrument panel. It may be displayed by a device (for example, a multifunction display 142, a navigation device display 143, etc.).
Further, for example, a head-up display device may be used to project the image onto the windshield.
(3) In the embodiment, in the automatic driving control that automatically changes lanes, a side situation image is displayed when waiting for a lane change, but the side is waiting for a steering operation other than a lane change. The lane status image may be displayed. Further, the area and the field of view included in the lateral situation image are not particularly limited.
Further, the present invention can be applied not only to automatic driving control but also to driving support control for supporting steering operation related to lane change, for example, in response to a lane change operation (for example, lighting of a turn signal lamp) by a driver. Is.
(4) In the embodiment, the display of the side situation image I2 is terminated according to the end of the steering standby state, but instead of such a configuration, the side situation image I2 is terminated according to the end of the steering operation. May be configured to end the display of.

1 Image display device 10 Rear side image processing unit 11 Rear camera 12 Left rear side camera 13 Right rear side camera 20 Stereo camera processing unit 21 Left front camera 22 Right front camera 30 Environment recognition unit 31 Left rear side radar 32 Right rear side radar 40 Driver monitoring unit 41 Driver monitoring camera 50 Display image generation unit 60 Display control unit 70 LCD
110 Windshield 120 Front door glass 121 Main body 122 Triangular window 130 Side mirror 140 Instrument panel 141 Combination meter 142 Multi-function display 143 Navigation device display 150 Steering wheel 160 A pillar 170 Roof 171 Stereo camera housing 180 Room mirror 200 Automatic Operation control unit 201 Input / output device 210 Engine control unit 220 Transmission control unit 221 Forward / backward switching actuator 222 Range detection sensor 230 Behavior control unit 231 Hydraulic control unit (HCU)
232 Vehicle speed sensor 240 Electric power steering (EPS) control unit 241 Motor 242 Steering angle sensor I1 Normal display image I2, I3 Side situation image 310 Own vehicle 320 Own vehicle driving lane 330 Adjacent lane 340 Leading vehicle 350 Target driving trajectory display 360 Subsequent vehicle

Claims (4)

An environment recognition means that recognizes the environment around the vehicle,
An image display device provided in a vehicle provided with a driving control means that automatically performs a steering operation based on the recognition result of the environment recognition means.
Image display means placed in front of the occupants,
When the operation control means is in the steering standby state waiting for the steering operation according to the recognition result of the environment recognition means, a side situation image displaying the lateral situation in the planned steering direction with respect to the own vehicle is displayed. A display control means for displaying on the image display means is provided.
Said lateral status image, together includes within the display range at least part of the neighbor lane position in the longitudinal direction of the vehicle that put the range of the vehicle and the lap, the side of the vehicle the environment recognizing means recognizes An image display device characterized in that the side view of another vehicle running in parallel is displayed together with a line indicating the positions of the front end portion and the rear end portion of the own vehicle. The steering operation is for the vehicle performs a lane change from the lane currently driving to the adjacent lane,
The image display device according to claim 1, wherein the side situation image displays a side situation on the adjacent lane side with respect to the own vehicle. The side situation image is characterized in that the display is started according to the start of the steering standby state of the operation control means and the display is terminated according to the end of the steering standby state. The image display device according to claim 2. 1. Item 2. The image display device according to item 2.

Images