JP2018173717A - Image display device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image display device capable of easily checking a sideward state in automatic steering standby.SOLUTION: An image display device 1 to be provided in a vehicle includes: environment recognition means 30 for recognizing an environment in the periphery of a self-vehicle; and driving control means 200 for automatically performing a steering action on the basis of a recognition result of the environment recognition means. The device includes: image display means 70 placed in front of an occupant; and display control means 50, 60 for allowing the image display means to display a sideward state image I2 for displaying a sideward state of a steering planned direction with respect to the self-vehicle when the driving control means is in a steering standby state to wait for a steering action in accordance with a recognition result of the environment recognition means.SELECTED DRAWING: Figure 5

Description

  The present invention relates to an image display device provided in an automobile that changes lanes by automatic steering, and more particularly, to an apparatus that can easily check a side situation during standby for automatic steering.

In a vehicle such as an automobile, an image that displays an image captured by a camera provided on the side part of the vehicle body, the rear part, or the like, instead of or in combination with existing mirrors, as rear side monitoring means of the vehicle It has been proposed to use a display device (camera monitoring system).
As a conventional technique related to such an image processing apparatus, for example, Patent Document 1 discloses image display in an inner region and an outer region of a steering wheel rim portion in front view of a driver, and each captured image straddles a rim portion. What is configured to move is described.

In recent years, driving assistance devices and automatic driving devices that recognize the environment around the host vehicle and automatically perform a steering operation such as lane maintenance or lane change are becoming widespread in vehicles such as automobiles.
In Patent Document 2, in an automatic driving support system, when it is determined that automatic driving control cannot be performed at the present time or in the future during traveling in an automatic driving section in which automatic driving control of a vehicle is permitted, automatic driving control is performed. It is described that the reason for non-execution, which is the reason why cannot be implemented, is acquired, automatic driving control cannot be performed, and the obtained reason for non-execution is also guided by a character string or voice.

Patent No. 5903319 JP 2006-28927 A

When a lane change is made from a lane in which the host vehicle is traveling to an adjacent lane by a driving support device or an automatic driving device having an automatic steering function, for example, an obstacle such as another vehicle is approaching from the rear side, etc. For this reason, it may be necessary to wait for the steering operation for changing the lane.
In such a case, as described in Patent Document 2, for example, it is conceivable to guide the reason for waiting for the steering operation by characters, voice, or the like. It is necessary to look back from the side and confirm it visually, and it is required that the user (occupant) can grasp the reason why the vehicle is in a steering standby state more intuitively and easily.
In view of the above-described problems, an object of the present invention is to provide an image display device that can easily check a side situation during standby for automatic steering.

The present invention solves the above-described problems by the following means.
The invention according to claim 1 is an image display provided in a vehicle comprising environment recognition means for recognizing the environment around the host vehicle and driving control means for automatically performing a steering operation based on the recognition result of the environment recognition means. An image display means disposed in front of an occupant, and when the operation control means is in a steering standby state in which the steering operation waits according to a recognition result of the environment recognition means. An image display device comprising: a display control unit that causes the image display unit to display a side state image that displays a side state in a steering direction.
According to this, when a steering standby state occurs, by displaying the situation of the side in the planned steering direction, the user can immediately check the obstacle causing the steering standby without looking back It is possible to intuitively and easily grasp the reason why the steering standby is performed and the prospect of the completion of the steering standby.

In the invention according to claim 2, the steering operation is to change the lane from the lane in which the host vehicle is currently traveling to the adjacent lane, and the side situation image is on the side of the adjacent lane with respect to the host vehicle. The image display device according to claim 1, wherein a state of a side of the image is displayed.
According to this, when the lane change is automatically performed by the operation control device, the above-described effects can be reliably obtained.

The invention according to claim 3 is characterized in that the side situation image includes at least a part of the adjacent lane in a range where the position of the host vehicle in the front-rear direction overlaps with the host vehicle in a display range. The image display device according to claim 1.
According to this, it is possible to appropriately grasp the cause of the steering standby by displaying the situation of the area where there is a risk of collision with an obstacle when the host vehicle moves in the lateral direction.

The invention according to claim 4 is characterized in that the side situation image includes a range corresponding to a field of view by a monitoring device that monitors a rear side of the host vehicle during normal driving in a display range. The image display device according to any one of claims 1 to 3.
According to this, the user can intuitively grasp the cause of the steering standby by using the side situation image as a field of view familiar to the user similar to the side mirror or camera monitoring system, for example.

The invention according to claim 5 is characterized in that the side situation image includes other vehicles recognized by the environment recognition means in a display range. This is an image display device.
According to this, when the steering standby is performed due to a risk of contact with another vehicle, the reason for the steering standby can be appropriately recognized by displaying the other vehicle.

In the invention according to claim 6, the display of the side situation image is disclosed in response to the start of the steering standby state of the driving control means, and the display is ended in accordance with the end of the steering standby state. The image display device according to any one of claims 1 to 5, wherein
According to a seventh aspect of the present invention, the display of the side situation image is started in response to the start of the steering standby state of the driving control means and the display is ended in response to the end of the steering operation. 6. The image display device according to claim 1, wherein the image display device is characterized in that
According to each of these inventions, it is possible to return to the original display at an appropriate timing after displaying the side situation image for a sufficient period to inform the user of the cause of the steering standby.

  As described above, according to the present invention, it is possible to provide an image display device capable of easily confirming a side situation during standby for automatic steering.

It is a block diagram which shows the structure of embodiment of the image display apparatus to which this invention is applied. It is a typical perspective view which shows arrangement | positioning of the rear side camera etc. in the image display apparatus of embodiment. It is a figure which shows an example of the user visual field in the vehicle which has the image display apparatus of embodiment. It is a block diagram which shows the structure of the driving | running control apparatus provided in the vehicle which has the image display apparatus of embodiment. It is a flowchart which shows operation | movement of the image display apparatus of embodiment. It is a figure which shows an example of the display image at the time of normal in the image display apparatus of embodiment. It is a figure which shows an example of the display image at the time of the lane change waiting | standby in the image display apparatus of embodiment. It is a figure which shows the other example of the display image at the time of the lane change waiting | standby in the image display apparatus of embodiment.

Hereinafter, embodiments of an image display device to which the present invention is applied will be described.
The image display apparatus according to the embodiment is a camera monitoring system that is provided in an automobile such as a passenger car, for example, and presents an image or the like obtained by capturing the rear side of the vehicle body with a camera to a user such as a driver.

FIG. 1 is a block diagram illustrating a configuration of an image display apparatus according to an embodiment.
The image display apparatus 1 includes a rear side image processing unit 10, a rear camera 11, a left rear side camera 12, a right rear side camera 13, a stereo camera processing unit 20, a left front camera 21, a right front camera 22, and environment recognition. The unit 30 includes a left rear side radar 31, a right rear side radar 32, a driver monitoring unit 40, a driver monitoring camera 41, a display image generation unit 50, a display control unit 60, an LCD 70, and the like.

The rear side image processing unit 10 is a digital signal processing device that processes output signals of the rear camera 11, the left rear side camera 12, and the right rear side camera 13 and outputs them as image data.
The video signal output from the rear side image processing unit 10 is transmitted to the environment recognition unit 30 and the display image generation unit 50.
The rear camera 11, the left rear side camera 12, and the right rear side camera 13 shoot a moving image of the situation around the vehicle and output a video signal in real time. A solid-state imaging device such as a CMOS, its driving device, and lens Etc.

FIG. 2 is a schematic perspective view illustrating the arrangement of the rear side camera and the like in the image display apparatus according to the embodiment.
The rear camera 11 is disposed at the center in the vehicle width direction at the rear end of the vehicle body B with the optical axis direction facing the vehicle rear side.
The left rear side camera 12 is arranged on the left side surface portion of the vehicle body B with the optical axis direction facing the rear side and the vehicle width direction outside.
The left rear side camera 12 is disposed at a location adjacent to the side mirror 130, for example.
The field of view (imagingable range) F12 of the left rear side camera 12 partially wraps with the field of view F11 of the rear camera 11, and the entire range from the rear to the left side of the host vehicle is substantially determined by at least one of the cameras. The image is taken so that there is no blind spot.
The right rear side camera 13 (not shown in FIG. 2) is disposed substantially symmetrically with the left rear side camera 12.

The stereo camera processing unit 20 performs known stereo camera processing on the images captured by the left front camera 21 and the right front camera 22, respectively, and detects an object ahead of the host vehicle and its distance from the host vehicle.
The output of the stereo camera processing unit 20 is transmitted to the environment recognition unit 30.
The left front camera 21 and the right front camera 22 capture a moving image of the situation in front of the vehicle and output a video signal in real time. The left front camera 21 and the right front camera 22 include a solid-state imaging device such as a CMOS, a driving device thereof, a lens and the like.
The left front camera 21 and the right front camera 22 are disposed, for example, in the vicinity of the upper end portion of the front window glass in the vehicle interior and spaced apart in the left-right direction.

The environment recognition unit 30 recognizes the environment around the host vehicle based on information provided from the rear side image processing unit 10, the stereo camera processing unit 20, the left rear side radar 31, the right rear side radar 32, and the like. To do.
The environment recognition unit 30 functions as the other vehicle detection means according to the present invention in cooperation with the left rear side radar 31 and the right rear side radar 32.
Here, as recognized environments, the lane shape of the own vehicle traveling lane in which the own vehicle travels, the lane shape of the adjacent lane adjacent to the own vehicle traveling lane, other vehicles traveling around the own vehicle, pedestrians, buildings (For example, buildings, guardrails, signs, traffic lights, bridges, tunnels, median strips, roadside strips, etc.).

The left rear side radar 31 and the right rear side radar 32 are for detecting an object such as another vehicle existing behind the host vehicle.
As the left rear side radar 31 and the right rear side radar 32, for example, a millimeter wave radar can be used.
The left rear side radar 31 and the right rear side radar 32 can detect the relative position of the other vehicle with respect to the host vehicle when the other vehicle exists, and based on the transition of the relative position, the other vehicle It is possible to calculate the relative speed with respect to the own vehicle.
The left rear side radar 31 and the right rear side radar 32 are arranged on the left and right (for example, inside the rear bumper) at the rear of the vehicle body.

The driver monitoring unit 40 detects the posture, behavior, line-of-sight direction, etc. of the user (driver during manual operation) based on the image captured by the driver monitoring camera 41.
The driver monitoring camera 41 is provided in front of the driver in the passenger compartment.

The display image generation unit 50 generates a display image to be displayed on the LCD 70 based on inputs from the rear side image processing unit 10, the environment recognition unit 30, and the driver monitoring unit 40.
The function of the display image generation unit 50 and an example of the generated display image will be described in detail later.
The display image may be a real image obtained by performing predetermined image processing on the video imaged by the rear camera 11, the left rear side camera 12, and the right rear side camera 13, and corresponds to the real image. It may be computer graphics or the like for displaying information. Further, the photographed image and computer graphics may be combined and used.

The display control unit 60 is a display driving device that causes the LCD 70 to display the display image generated by the display image generation unit 50.
The LCD 70 is a display unit that is provided in front of the user (occupant / driver during manual operation) and displays a display image to the user.
The arrangement of the LCD 70 in the vehicle interior will be described in detail later.

FIG. 3 is a diagram illustrating an example of a user field of view in a vehicle including the image display device according to the embodiment.
As shown in FIG. 3, the vehicle includes a windshield 110, a front door glass 120, a side mirror 130, an instrument panel 140, a steering wheel 150, an A pillar 160, a roof 170, a room mirror 180, and the like.

The windshield 110 is disposed on the front side of the driver.
The windshield 110 is a secondary curved glass that is formed in a substantially horizontally long rectangular shape and is curved in a direction in which the front is convex.
The windshield 110 is disposed so as to be tilted rearward so that the upper end portion is on the vehicle rear side with respect to the lower end portion.

The front door glass 120 is provided in the upper part of the left and right front doors used for passengers getting on and off and on the side of the driver.
The front door glass 120 includes a liftable main body 121 and a fixed triangular window 122 provided at the front of the main body 121.

The side mirror 130 is used by the driver to check the left / right rear view.
The side mirror 130 protrudes outward in the vehicle width direction from the outer panels of the left and right front doors.
In the user field of view, for example, the side mirror 130 is visible in the vicinity of the front end portion of the main body 121 of the front door glass 120.

The instrument panel 140 is an interior member provided below the windshield 110 in the vehicle interior.
The instrument panel 140 also functions as a housing that accommodates various instruments, display devices, switches, air conditioners, front passenger airbag devices, knee protection airbag devices, and the like.
The instrument panel 140 includes a combination meter 141, a multi-function display 142, a navigation device display 143, and the like.

The combination meter 141 is provided in front of the driver's seat and is a unit of various instruments such as a speedometer, an engine tachometer, and a distance meter.
The combination meter 141 has a built-in LCD 70.
The multi-function display 142 is image display means such as an LCD provided at the upper part of the instrument panel 140 in the center in the vehicle width direction.
The navigation device display 143 is provided at the lower portion of the instrument panel 140 in the center in the vehicle width direction.

The steering wheel 150 is an annular operation member that inputs a steering operation when the driver manually operates.
The steering wheel 150 is provided substantially in front of the driver.
The combination meter 141 is visible from the inner diameter side of the upper half of the steering wheel 150 in the user view.

The A-pillar 160 is a columnar vehicle body structural member disposed along the side end portion of the windshield 110 and the front end portion of the front door glass 120.
The surface of the A pillar 160 on the vehicle interior side is covered with a resin pillar trim.

The roof 170 is formed to extend rearward from the upper end portion of the windshield 110.
A surface portion of the roof 170 on the vehicle interior side is covered with a resin roof trim.
A stereo camera housing portion 171 in which the left front camera 21 and the right front camera 22 are housed is provided at the front end portion of the roof 170 in the vehicle width direction central portion.

The rearview mirror 180 is a rearward confirmation mirror provided in the vehicle interior.
The room mirror 180 is provided in the vicinity of the upper end portion of the windshield 110 in the vehicle width direction central portion via a stay (not shown).

The vehicle includes an operation control device described below.
This driving control device constitutes an automatic driving system having an automatic driving function for automatically performing driving operations such as steering and acceleration / deceleration according to the recognition result by the environment recognition unit 30.
FIG. 4 is a block diagram illustrating a configuration of an operation control device provided in a vehicle having the image display device of the embodiment.
As shown in FIG. 4, the operation control device includes an automatic operation control unit 200, an engine control unit 210, a transmission control unit 220, a behavior control unit 230, an electric power steering (EPS) control unit 240, and the like.
Each unit described above is configured as a unit having information processing means such as a CPU, storage means such as RAM and ROM, an input / output interface, a bus connecting these, and the like. These units can communicate with each other via an in-vehicle LAN system such as a CAN communication system.

  When the automatic operation mode is selected by the user, the automatic operation control unit 200 outputs a control command to the engine control unit 210, the transmission control unit 220, the behavior control unit 230, the EPS control unit 240, etc. The automatic operation control for running the vehicle automatically is executed.

When the automatic driving mode is selected, the automatic driving control unit 200 advances the own vehicle according to information about the situation around the own vehicle provided from the environment recognition unit 30 and a command from a driver (not shown). Set the target driving locus.
The automatic operation control unit 200 automatically performs acceleration (start), deceleration (stop), forward / reverse switching, steering, etc. of the vehicle so that the actual travel locus of the host vehicle substantially matches the target travel locus. Then, automatic driving is executed in which the vehicle automatically travels to a preset destination.
In addition, the automatic operation mode is a manual operation in which a manual operation by a driver is performed when the user desires a manual operation or when it is difficult to continue the automatic operation in response to a predetermined release operation from the user. Return to operation mode is possible.
In addition, the automatic operation control unit 200 can communicate with the display image generation unit 50 and gives an instruction to the display image generation unit 50 to switch the mode of the image to be generated. This point will be described in detail later.

An input / output device 201 is connected to the automatic operation control unit 200.
The input / output device 201 outputs information such as warnings and various messages to the user from the automatic operation control unit 200 and accepts input of various operations from the user.
The input / output device 201 includes, for example, an image display device such as an LCD, an audio output device such as a speaker, and an operation input device such as a touch panel.
For example, the input / output device 201 can be used as the touch panel such as the LCD 70 and the multi-function display 142.

The engine control unit 210 comprehensively controls an engine that is a power source for driving the vehicle and its auxiliary equipment.
For example, a 4-stroke gasoline engine is used as the engine.
The engine control unit (ECU) 210 can control the output torque of the engine by controlling the throttle valve opening, fuel injection amount and injection timing, ignition timing, and the like of the engine.
In a state where the vehicle is driven in accordance with the driving operation of the driver, the engine control unit 210 outputs the engine so that the actual torque of the engine approaches the driver required torque set based on the operation amount of the accelerator pedal. To control.
Further, when the vehicle performs automatic driving, the engine control unit 210 controls the output of the engine in accordance with a command from the automatic driving control unit 200.

The transmission control unit (TCU) 220 shifts the rotational output of the engine, and comprehensively controls a transmission and auxiliary equipment (not shown) that switch between forward and reverse of the vehicle.
When the vehicle performs automatic driving, the transmission control unit 220 performs range switching such as forward / backward travel and setting of the gear ratio in response to a command from the automatic driving control unit 200.
As the transmission, for example, chain-type, belt-type, toroidal-type CVT, and various automatic transmissions such as a step AT having a plurality of planetary gear sets, a dual clutch transmission, and an AMT can be used.
In addition to a transmission mechanism such as a variator, the transmission includes a starting device such as a torque converter, a dry clutch, a wet clutch, and a forward / reverse switching mechanism that switches between a forward travel range and a reverse travel range. Yes.

The transmission control unit 220 is connected to a forward / reverse switching actuator 221, a range detection sensor 222, and the like.
The forward / reverse switching actuator 221 drives a forward / reverse switching valve for switching an oil passage that supplies hydraulic pressure to the forward / reverse switching mechanism, and switches forward / backward travel of the vehicle.
The forward / reverse switching actuator 221 is an electric actuator such as a solenoid, for example.
The range detection sensor 222 is a sensor (switch) that determines whether the currently selected range in the transmission is for forward movement or reverse movement.

The behavior control unit 230 individually controls the wheel cylinder hydraulic pressures of the hydraulic service brakes provided on the left and right front and rear wheels, respectively, to control the vehicle behavior such as understeer and oversteer, and wheel lock during braking. Anti-lock brake control to recover
A hydraulic control unit (HCU) 231 and a vehicle speed sensor 232 are connected to the behavior control unit 230.

The HCU 231 includes an electric pump that pressurizes brake fluid that is a working fluid of a hydraulic service brake, a valve that individually adjusts the hydraulic pressure supplied to the wheel cylinder of each wheel, and the like.
When the vehicle performs automatic driving, the HCU 231 generates braking force on the wheel cylinder of each wheel in response to a braking command from the automatic driving control unit 200.
The vehicle speed sensor 232 is provided at the hub portion of each wheel, and generates a vehicle speed pulse signal having a frequency proportional to the rotational speed of the wheel.
It is possible to calculate the traveling speed (vehicle speed) of the vehicle by detecting the frequency of the vehicle speed pulse signal and performing a predetermined calculation process.

The electric power steering (EPS) control unit 240 comprehensively controls an electric power steering device that assists a steering operation by a driver with an electric motor and its auxiliary devices.
The EPS control unit 240 is connected to a motor 241, a steering angle sensor 242, and the like.

The motor 241 is an electric actuator that applies an assist force to the steering system of the vehicle to assist the steering operation by the driver, or changes the steering angle during automatic driving.
When the vehicle performs automatic driving, the motor 241 applies torque to the steering system in accordance with the steering command from the automatic driving control unit 200 so that the steering angle of the steering system approaches a predetermined target steering angle. Turn the steering.
The steering angle sensor 242 detects the current steering angle in the vehicle steering system.
The steering angle sensor 242 includes, for example, a position encoder that detects the angular position of the steering shaft.

In the image display apparatus 1 according to the embodiment, after the automatic driving control unit 200 sets a target travel locus including a lane change, for example, due to an approach of an obstacle such as another vehicle, the steering standby waits for a steering operation. An image obtained by capturing the rear side of the host vehicle (side situation image I2) is displayed so that the user can confirm the obstacle that caused the problem when the state is reached.
This will be described in more detail below.
FIG. 5 is a flowchart showing the operation of the image display apparatus according to the embodiment, and shows a state when the automatic driving control unit 200 sets a target travel locus including a lane change accompanied by a steering operation.
Hereinafter, the steps will be described step by step.

<Step S01: Target travel locus setting>
The automatic driving control unit 200 sets the target travel locus of the own vehicle based on the environment recognition result of the environment recognition unit 30 and the input from the user.
Thereafter, the process proceeds to step S02.

<Step S02: Steering Plan Presence / Absence>
The automatic driving control unit 200 determines whether or not it is necessary to perform a steering operation of a predetermined steering angle or more when the host vehicle travels along the target travel locus set in step S01.
If a steering operation of a predetermined steering angle or more is necessary, the process proceeds to step S03 as being scheduled for steering, and the series of processes is terminated (returned) as if there is no scheduled steering otherwise.

<Step S03: Steering direction environment recognition>
The environment recognition unit 30 newly recognizes the environment in the direction in which the steering is scheduled, and detects an obstacle that is newly recognized after the target travel locus is set or an obstacle that is approaching.
For example, when the target travel locus is accompanied by a lane change, the following vehicle traveling in the changed lane is detected by the left and right rear side cameras 12, 13, the rear side radars 31, 32, etc. .
Thereafter, the process proceeds to step S04.

<Step S04: Steering standby determination>
Based on the recognition result of the environment recognition unit 30 in step S03, the automatic driving control unit 200 determines whether or not the steering standby for temporarily waiting for the steering operation is necessary.
For example, when the target travel locus includes a lane change and a subsequent vehicle traveling in the changed lane is approaching at a predetermined relative speed or more, it is determined that the contact risk is high, and the automatic driving control unit 200 Then, it is determined that the steering standby (lane change standby) is performed until the succeeding vehicle passes the host vehicle and a space on the side of the host vehicle becomes free.
When the steering standby is necessary, the process proceeds to step S05, and when the steering standby is not necessary (when the steering operation is possible according to the original target travel locus), the process proceeds to step S09.

<Step S05: Side status image display start>
The automatic driving control unit 200 instructs the display image generation unit 50 to display a side situation image I2 (see FIG. 7) that is an image showing the situation behind the host vehicle.
The display image generation unit 50 generates image data related to the side situation image I2, provides it to the display control unit 60, and causes the LCD 70 to display it.
In the state where the side situation image I2 is not displayed, the normal display image I1 (see FIG. 6) is displayed on the LCD 70.
The normal display image I1 and the side situation image I2 will be described in detail later.
Thereafter, the process proceeds to step S06.

<Step S06: Steering standby end determination>
The automatic driving control unit 200 determines whether or not the steering standby state has ended.
The determination of the end of the steering standby state is made based on the situation on the side of the host vehicle that the environment recognition unit 30 recognizes sequentially.
For example, when the other vehicle of the adjacent vehicle completes overtaking the own vehicle and a space on the side of the own vehicle is free, the end of the steering standby is determined.
When the steering standby state ends, the process proceeds to step S07, and when the steering standby state does not end, step S06 is repeated at a predetermined time interval.

<Step S07: Start of steering operation>
The automatic operation control unit 200 issues a command to the EPS control unit 240 and starts giving a steering angle (steering operation) to the steering system by the motor 241.
Thereafter, the process proceeds to step S08.

<Step S08: End of Side Status Image Display>
The automatic operation control unit 200 causes the display image generation unit 50 to finish generating the side situation image I2, and returns the LCD 70 to a state in which the normal display image I1 is displayed.
Thereafter, the series of processing is terminated (returned).

<Step S09: Start of steering operation>
The automatic operation control unit 200 issues a command to the EPS control unit 240 and starts giving a steering angle (steering operation) to the steering system by the motor 241.
Thereafter, the series of processing is terminated.

Next, a display image on the LCD 70 in the present embodiment will be described.
The display image described below is generated by the display image generation unit 50.
FIG. 6 is a diagram illustrating an example of a normal display image in the image display apparatus according to the embodiment.
FIG. 7 is a diagram illustrating an example of a display image when waiting for a lane change in the image display apparatus according to the embodiment.

During normal times (except during steering standby), as shown in FIG. 6, a normal-time display image I1 that is an overhead view (bird's eye view) viewed from the rear side and obliquely above the host vehicle is displayed.
In the normal display image I1, for example, the host vehicle 310, the host vehicle travel lane 320, the adjacent lane 330, the preceding vehicle 340, the target travel locus display 350, and the like are displayed.
Such an overhead view is obtained by combining the images captured by the cameras with predetermined image processing, or in the display image generation unit 50 based on the environment recognition result of the environment recognition unit 30. Generated by rendering as graphics.
Further, instead of such a bird's-eye view (bird's-eye view), for example, a plan view showing a state in which the host vehicle and its periphery including substantially all directions are viewed from directly above may be used.

FIG. 6 shows a state in which the host vehicle 310 is traveling on a high-standard road (such as a highway) having two lanes on one side having the host vehicle traveling lane 320 and the adjacent lane 330 running in parallel in the same direction.
The host vehicle 310 is traveling in the host vehicle travel lane 320.
The adjacent lane 330 is adjacent to the right side of the host vehicle traveling lane 320 and is provided in parallel.
In the own vehicle travel lane 320, a preceding vehicle 340 is traveling in the same direction as the own vehicle 310 in front of the own vehicle 310.

When the automatic driving control unit 200 sets a target travel locus that overtakes the preceding vehicle 340 in order to avoid the preceding vehicle 340, the host vehicle 310 changes the lane from the host vehicle travel lane 320 to the adjacent lane 330 in the normal display image I1. A target travel locus display 350 indicating that the operation is to be performed is displayed.
The user recognizes that the automatic driving control unit 200 is about to change the lane based on the appearance of the target travel locus display 350 and the start of blinking of a turn signal (winker) lamp (not shown).

In FIG. 7, a side situation image I <b> 2 indicating the situation of the right rear side of the host vehicle 310 captured by the right rear side camera 13 is displayed.
The side situation image I2 includes at least a part of the adjacent lane 330 in a range in which the position of the host vehicle 310 in the front-rear direction overlaps with the host vehicle within the display range, and can cover a field of view substantially equivalent to the side mirror 130. It has become.
Such a side situation image I2 is, for example, a real image obtained by performing image processing such as adjustment of brightness, luminance, contrast, color tone, etc. on the image captured by the right rear side camera 13 and correction of distortion caused by lens aberration. Images can be used.
Alternatively, an image captured by the right rear side camera 13 or computer graphics rendered based on the output of the right rear side radar 32 may be used.
In the embodiment, the side situation image is a left-right reverse image because it does not give a sense of incongruity to the appearance when the rear side is visually recognized using a general side mirror 130.

In FIG. 7, in addition to the own vehicle 310, the own vehicle travel lane 320, and the adjacent lane 330, a subsequent vehicle 360 that is traveling in the adjacent lane 330 is displayed.
The following vehicle 360 travels in the same direction at a higher speed than the host vehicle 310 in the adjacent lane 330.

FIG. 8 is a diagram illustrating another example of a display image when waiting for a lane change in the image display device of the embodiment.
In FIG. 8, a super-wide-angle camera that can image the side of the host vehicle as the right rear side camera 13 and is arranged with the optical axis direction substantially along the vehicle width direction is used. A side situation image I3 indicating the situation is displayed.
In FIG. 8, the following vehicle 360 is substantially displayed as a side view.
The side display image I3 displays a front end line Lf indicating the front-rear direction position of the front end of the host vehicle and a rear end line Lr indicating the front-rear position of the rear end of the host vehicle. Based on the positional relationship with Lf and Lr, it can be determined whether or not the parallel running state in overtaking or the like has ended.
The side display images I2 and I3 can be selected and set, for example, which image is displayed in advance by the user.
Moreover, it is good also as a structure which switches sequentially from the side display image I2 to the side display image I3 according to the approach with respect to the own vehicle 310 of the following vehicle 360. FIG.

If the autonomous driving control unit 200 determines that there is a concern about interference (followed collision, etc.) with the following vehicle 360 when the host vehicle 310 has traveled faithfully to the initially set target traveling locus, Until the overtaking of the host vehicle 310 by 360 is completed, the traveling of the host vehicle traveling lane 320 is continued in the steering standby state.
Based on the side situation images I2 and I3 displayed on the LCD 70 arranged in front of the user without looking back at the rear side, the user can intuitively understand that the cause of the steering standby state is in the following vehicle 360. It can be recognized easily and easily, and it can be predicted how long the steering standby state will continue in the future by looking at the position and speed of the following vehicle 360 (the approaching state to the host vehicle).

According to the embodiment described above, the following effects can be obtained.
(1) When the steering standby state occurs, by displaying the side situation in the planned steering direction as the side situation image I2, the user can cause the steering standby without looking back. Obstacles such as the vehicle 360 can be immediately confirmed, and the reason why the steering standby is performed and the prospect of the completion of the steering standby can be intuitively and easily grasped.
(2) The steering operation is related to the lane change, and the side situation image I2 displays the situation of the adjacent lane 330 that is the lane change destination. The obtained effect can be obtained with certainty.
(3) The side situation image I2 includes at least a part of the adjacent lane 330 in a range where the position of the host vehicle 310 in the front-rear direction overlaps with the host vehicle, so that the host vehicle 310 moves in the lateral direction. It is possible to display the status of the area where there is a collision risk with the following vehicle 360 and the like, and to appropriately grasp the cause of the steering standby.
(4) By including the range corresponding to the field of view of the side mirror 130 in the side status image I2 in the display range, the user can intuitively grasp the cause of the steering standby from the familiar field of view.
(5) When the steering standby is performed due to the risk of contact with the following vehicle 360, the reason for the steering standby can be appropriately recognized by displaying the subsequent vehicle 360 in the side situation image I2.
(6) The side situation image I2 is displayed in response to the start of the steering standby state, and the display is ended in response to the end of the steering standby state. After the side situation image I2 is displayed for a sufficient period to notify, it can be returned to the normal display image I1 at an appropriate timing.

(Modification)
The present invention is not limited to the embodiments described above, and various modifications and changes are possible, and these are also within the technical scope of the present invention.
(1) The configuration of the image display device and the vehicle on which the image display device is mounted is not limited to the above-described embodiment, and can be changed as appropriate.
For example, the quantity and arrangement of cameras and radars can be changed as appropriate. In the embodiment, one rear camera and one left and right rear side camera are provided, but a plurality of rear cameras may be provided. Further, for example, a single wide-angle camera may be used to serve as a rear camera and a rear side camera.
(2) In the embodiment, the side situation image is displayed on the LCD 70 provided in front of the user (driver during manual operation). However, the display is not limited to this and is provided in the center of the instrument panel. It may be displayed on a device (for example, a multifunction display 142, a navigation device display 143, etc.).
Moreover, you may make it project on a windshield, for example using a head-up display apparatus.
(3) In the embodiment, in the automatic driving control for automatically changing the lane, the side situation image is displayed at the time of waiting for the lane change. A situation image may be displayed. Moreover, the area | region and visual field contained in a side condition image are not specifically limited, either.
The present invention is not limited to automatic driving control, and can be applied to driving support control that supports steering operation related to lane change or the like according to a lane changing operation (for example, lighting of a turn signal lamp) by a driver, for example. It is.
(4) In the embodiment, the display of the side situation image I2 is ended according to the end of the steering standby state, but instead of such a configuration, the side situation image I2 is changed according to the end of the steering operation. The display may be terminated.

DESCRIPTION OF SYMBOLS 1 Image display apparatus 10 Rear side image processing unit 11 Rear camera 12 Left rear side camera 13 Right rear side camera 20 Stereo camera processing unit 21 Left front camera 22 Right front camera 30 Environment recognition unit 31 Left rear side radar 32 Right rear side radar 40 Driver monitoring unit 41 Driver monitoring camera 50 Display image generation unit 60 Display control unit 70 LCD
DESCRIPTION OF SYMBOLS 110 Front glass 120 Front door glass 121 Main body part 122 Triangular window part 130 Side mirror 140 Instrument panel 141 Combination meter 142 Multifunction display 143 Navigation apparatus display 150 Steering wheel 160 A pillar 170 Roof 171 Stereo camera housing part 180 Room mirror 200 Automatic Operation control unit 201 Input / output device 210 Engine control unit 220 Transmission control unit 221 Forward / reverse switching actuator 222 Range detection sensor 230 Behavior control unit 231 Hydraulic control unit (HCU)
232 Vehicle speed sensor 240 Electric power steering (EPS) control unit 241 Motor 242 Steering angle sensor I1 Normal display image I2, I3 Side situation image 310 Own vehicle 320 Own vehicle traveling lane 330 Adjacent lane 340 Preceding vehicle 350 Target traveling locus display 360 Following vehicle

Claims (7)

An environment recognition means for recognizing the environment around the vehicle,
An image display device provided in a vehicle, comprising: driving control means for automatically performing a steering operation based on a recognition result of the environment recognition means;
Image display means arranged in front of the occupant;
When the driving control unit is in a steering standby state in which it waits for the steering operation according to the recognition result of the environment recognition unit, a side state image that displays a side state of the steering planned direction with respect to the host vehicle is displayed. An image display device comprising: display control means for displaying on the image display means. The steering operation is to change the lane from the lane in which the host vehicle is currently traveling to the adjacent lane,
The image display device according to claim 1, wherein the side situation image displays a situation on a side of the adjacent lane side with respect to the own vehicle. The said side situation image contains at least one part of the said adjacent lane in the range which the position in the front-back direction of the own vehicle overlaps with the own vehicle in a display range. Image display device. The said side situation image includes a range corresponding to a field of view by a monitoring device that monitors the rear side of the host vehicle during normal driving of the vehicle in a display range. The image display device according to claim 1. The image display device according to any one of claims 1 to 4, wherein the side situation image includes another vehicle recognized by the environment recognition unit in a display range. The display of the side situation image is disclosed in accordance with the start of the steering standby state of the driving control means, and the display is ended in accordance with the end of the steering standby state. The image display device according to claim 5. The display of the side situation image is started according to the start of the steering standby state of the driving control means and is ended according to the end of the steering operation. Item 6. The image display device according to any one of Items 5 to 5.

Images