JP2021046094A - Vehicle exterior monitor device - Google Patents

Abstract

To provide a vehicle exterior monitor device that is improved.SOLUTION: A vehicle exterior monitor device 10 comprises: one or a plurality of photographing devices 11-13 which photograph a rear side of a vehicle 1; a display device 14 that displays a display image 41 so that an occupant in the vehicle 1 can visually recognize the image; a generating part 16 that cuts out images, depending on a travelling situation, from photographed images 31 by the one or the plurality of photographing devices 11-13 and generates display images 41 to be displayed on the display device 14; and a steering detecting part 21 that detects steering for changing a travelling direction of the vehicle 1 from a longitudinal direction of the vehicle 1. The generating part 16 acquires steering information from the steering detecting part 21, cuts out portions of the acquired photographed images 31, in a cut-out range 36 corresponding to the acquired steering information, and generates the display images 41 in which portions 39 at a steering side of the cut-out images 37 and portions 40 at the opposite side of the steering side, whose sizes are different, of the images are displayed.SELECTED DRAWING: Figure 4

Description

The present invention relates to an external monitoring device.

In vehicles such as automobiles, a rearview mirror provided in the passenger compartment, a door mirror provided on the side surface of the vehicle body, and the like have been conventionally used for rear-view confirmation.
In recent years, in vehicles, cameras have come to be used in place of rear-view mirrors and door mirrors such as these rear-view mirrors. The image of the rear of the vehicle captured by the camera is displayed on a display device provided in the passenger compartment.
By monitoring the conventional mirror in this way, the brightness of the image can be adjusted so that the moving body can be easily seen, and the antiglare treatment can be performed so as to weaken the light of the headlight of the moving body behind. ,can do. It can be expected that visibility can be improved by suppressing the influence of the environment outside the vehicle.
However, in the current monitoring device outside the vehicle, the display device only displays a fixedly set rear captured image by, for example, a side rear camera whose imaging direction is set in advance. In this case, the image displayed on the monitor can display the same range as the conventional rear mirror or door mirror. However, as was potentially the case with conventional rear mirrors and door mirrors, it cannot always be said that the display is such that the rear range that the occupant wants to visually confirm can be sufficiently visually recognized.

Jikkai Sho 63-194044 Japanese Unexamined Patent Publication No. 2007-49248

Therefore, for example, it is conceivable to use the techniques of Patent Documents 1 and 2 to switch and cut out a part of the captured image according to the traveling situation and display it on the display device.

However, even if the display range of the image displayed on the display device is switched, the occupant may not be able to fully grasp that the display range has been switched. Even if the display range is switched on a display device that is fixedly located in the passenger compartment, the occupant may not immediately notice it. As a result, even if it is displayed on the display device, the occupant can immediately see the relative direction and position of the moving object behind the vehicle in the switched display image with respect to the vehicle by visually recognizing the display image. Not always recognizable. In particular, the plurality of moving objects shown in the image displayed on the display device are displayed in a plane. The occupant may not always be able to easily see a distant moving object that appears small on the rear side in an image in which a nearby moving object appears large. The occupant must understand the display range of the switched display image by visually recognizing the entire display image, and grasp the relative direction and position of each moving object displayed on the display image. When visually recognizing a conventional rear mirror, for example, the occupant understands the display range of the rear mirror before that. In this case, the occupant can instantly grasp the moving body behind the vehicle by visually recognizing the rear mirror.

As described above, the monitoring device outside the vehicle is required to be improved.

The vehicle exterior monitor device according to the present invention includes one or a plurality of imaging devices that image the rear of the vehicle, a display device that visually displays a display image by the occupant of the vehicle, and an image captured by the one or a plurality of the imaging devices. A generation unit that cuts out an image according to the traveling situation and generates the display image to be displayed on the display device, and a steering detection unit that detects steering for changing the traveling direction of the vehicle from the front-rear direction of the vehicle. The generation unit acquires steering information from the steering detection unit, cuts out a part of the acquired captured image according to the cutting range according to the acquired steering information, and steers the cut out image. Generate a display image that displays the side part and the part on the opposite side of the steering in different sizes.

Preferably, the generation unit generates a display image so that the steering-side portion of the cut-out image is displayed larger than the steering-opposite portion.

Preferably, the generation unit generates a display image so that the height of the portion on the steering side in the display image is higher than the portion on the opposite side of the steering according to the steering information acquired from the steering detection unit. Therefore, it is preferable to display images having different heights on the left and right on the display device.

Preferably, in the display image, the height of the steering side portion is higher than the steering opposite portion as the steering amount acquired from the steering detection unit is larger.

Preferably, the generation unit divides the cut-out image into a central portion including the direction directly rearward of the vehicle and left and right right and left portions of the central portion, and determines the height of the right portion and the left portion. It is preferable that the display image is generated by processing so that the steering side is higher than the central portion and the opposite side of the steering is lower than the central portion.

Preferably, the height of the central portion including the direction directly behind the vehicle is the same as the height of the displayed image when the traveling direction of the vehicle is the front-rear direction of the vehicle.

Preferably, the generation unit displays the display image so that the display width of the portion on the steering side in the display image is wider than the display width of the portion on the opposite side of the steering according to the steering information acquired from the steering detection unit. Is generated, and it is preferable that the portion on the steering side is wider than the portion on the opposite side of the steering in the image displayed on the display device.

Preferably, the generation unit moves the cutout range of the acquired captured image in the steering direction according to the steering amount acquired from the steering detection unit.

Preferably, the generation unit maintains the cut-out range at the start of steering until the steering is completed, even if the steering information acquired from the steering detection unit changes during steering.

In the present invention, a generation unit that acquires an image captured by one or more imaging devices and generates a display image to be displayed on the display device obtains an image from the images captured by the one or more imaging devices according to a driving situation. Cut out and generate a display image to be displayed on the display device. As a result, the display device can display an image in a range that is expected to be confirmed by the occupant according to the driving situation.
In particular, the present invention includes a steering detection unit that detects steering for changing the traveling direction of the vehicle from the front-rear direction of the vehicle. Then, the generation unit acquires steering information from the steering detection unit, cuts out a part of the acquired captured image according to the cutting range according to the acquired steering information, and steers the cut-out image with the steering side part. Generate a display image that displays the opposite side in a different size. As a result, the portion on the steering side is displayed on the display device during steering in a size different from that on the portion on the opposite side of steering. For example, the steering-side portion of the clipped image is displayed larger than the steering-opposite portion. As a result, the occupant can distinguish the portion on the steering side from the portion on the opposite side of the steering simply by visually recognizing the displayed image. The occupant can easily grasp that the image on the steering direction side is visually confirmed and confirmed only by visually recognizing the part on the steering side of the displayed image, for example. The occupant can directly visually recognize the steering side part and immediately see the moving object, obstacle, road surface condition, etc. on the steering direction side without taking the procedure of visually recognizing the entire display image and visually understanding the display range. Can be grasped instantly.

FIG. 1 is an explanatory diagram of a running situation of an example of an automobile having an external monitoring device according to an embodiment of the present invention. FIG. 2 is an explanatory diagram of the arrangement of the display device of the monitoring device outside the vehicle in the vehicle interior. FIG. 3 is an explanatory diagram of an example of a display image that can be displayed on the display device of the monitoring device outside the vehicle in the driving situation of FIG. FIG. 4 is an explanatory diagram of an external monitoring device provided in the automobile of FIG. FIG. 5 is a flowchart of an outside monitor process for obtaining a display image from an captured image by the monitor ECU of FIG. FIG. 6 is an explanatory diagram of the image processed by FIG. FIG. 7 is an explanatory diagram of various modifications of the image processing of FIG. FIG. 8 is an explanatory view of a modified example of the outer shape of the display image displayed on the display device during steering. FIG. 9 is a flowchart of a modified example of the vehicle exterior monitor processing of FIG.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

FIG. 1 is an explanatory diagram of a running situation of an example of an automobile 1 having an external monitoring device 10 according to an embodiment of the present invention.
FIG. 1 shows an automobile 1 traveling in the central lane of a three-lane road that curves to the right. Behind the vehicle 1, there are another vehicle 9 traveling in the same lane and another vehicle 9 traveling in the right lane inside the curve of the roadway.
The automobile 1 of FIG. 1 has a vehicle body 2. A passenger compartment 3 on which an occupant such as a driver rides is formed in the central portion of the vehicle body 2.

Further, the automobile 1 of FIG. 1 has an external monitoring device 10. The vehicle exterior monitoring device 10 displays a display image 41 based on the captured image 31 captured by the imaging device that images the rear of the vehicle body 2 on the display device 14.

On the left and right side surfaces of the vehicle body 2, right side rear cameras 11 and left side rear cameras 12 are provided at the positions of the door mirrors instead of the door mirrors. The right side rear camera 11 and the left side rear camera 12 are installed so as to face the vehicle 1 in the oblique rear direction. As a result, the right side rear camera 11 can image the range in the right rear direction of the vehicle body 2 from the right side surface of the vehicle body 2 to the substantially right lateral direction. The left side rear camera 12 can capture a range in the left rear direction of the vehicle body 2 from the left side surface of the vehicle body 2 to a substantially left lateral direction.

A center rear camera 13 is provided in the center of the rear portion of the vehicle body 2, for example, the passenger compartment 3. The center rear camera 13 basically needs to be located at the Y0 position, which is the center of the vehicle body 2 in the left-right direction. It may be provided at the position of the mirror. The center rear camera 13 is installed so as to face directly behind the vehicle body 2. As a result, the center rear camera 13 can take an image of the rear range including the oblique rear direction of the vehicle body 2 with the direction directly behind the vehicle body 2 as the center. The imaging range of the center rear camera 13 overlaps with the imaging range of the right side rear camera 11 and the imaging range of the left side rear camera 12.

The monitoring device 10 outside the vehicle may have a 360-degree camera in place of or together with the plurality of imaging devices such as the right side rear camera 11, the left side rear camera 12, and the center rear camera 13. .. The 360 degree camera captures an image all around. The 360-degree camera may be arranged, for example, on the roof of the vehicle body 2 or at the rear of the vehicle body 2. As a result, the 360-degree camera can take an overall image of the rear of the vehicle body 2 from the right side to the left side.

FIG. 2 is an explanatory diagram of the arrangement of the display device 14 of the external monitoring device 10 in the vehicle interior 3.
FIG. 2 shows the inside of the front of the passenger compartment 3 of the automobile 1. Inside the front of the passenger compartment 3, a dashboard 5 is provided under the windshield 4.

The display device 14 of the external monitoring device 10 is one liquid crystal panel. It has a single plane display surface for displaying the display image 41 of the liquid crystal panel. The display device 14 may be provided side by side so that a plurality of liquid crystal panels form a single flat display surface.
The display device 14 is embedded in the dashboard 5 and is fixedly provided. As a result, the occupant of the vehicle interior 3 can visually recognize the rear of the automobile 1 by the display image 41 of the display device 14 arranged below the windshield 4 while checking the front of the automobile 1 through the windshield 4.
The display device 14 may be provided on the dashboard 5 so that its overall position can be finely adjusted according to, for example, the size of the occupant.

FIG. 3 is an explanatory diagram of an example of a display image 41 that can be displayed on the display device 14 of the monitoring device 10 outside the vehicle in the traveling situation of FIG.
FIG. 3A is a captured image 31 when the automobile 1 is at the position shown by the solid line in FIG. 1 and is traveling on a straight line portion about the roadway.
FIG. 3B is a captured image 31 when the automobile 1 is at the position shown by the dotted line in FIG. 1 and is traveling on a portion of the roadway that curves to the right.
These captured images 31 may be images obtained by synthesizing images captured by a plurality of imaging devices such as the right side rear camera 11, the left side rear camera 12, and the center rear camera 13, but may be simply captured by the center rear camera 13 or the 360 degree camera. It may be the image itself captured by one imaging device.

Usually, the monitoring device 10 outside the vehicle may cut out a fixed fixed portion from the captured image 31 and cut out a display image 41 as shown by a dotted line frame in FIGS. 3 (A) and 3 (B). In this case, the display image 41 is a fixed cut-out image of a fixed range extending to the left and right with the Y0 position of the vehicle body 2 as the center, regardless of the traveling state of the automobile 1. Therefore, as shown in FIG. 3B, when the automobile 1 is traveling on a curve, the display image 41 is an image closer to the outside of the curve. Such an image can be displayed in the same range as the conventional rear mirror and door mirror, but the occupant always wants to visually confirm it, as was potentially in the conventional rear mirror and door mirror. It cannot be said that the area behind the vehicle is displayed so that it can be sufficiently visually recognized. For example, another automobile 9 overtaking from the inside of the curve cannot be captured in the display image 41 and displayed satisfactorily.

Therefore, it is conceivable to switch and display the image to be displayed on the display device 14 of the external monitoring device 10 according to, for example, the traveling situation of the automobile 1. For example, it is conceivable that the display device 14 displays the display image 41 in which the range is moved in the same direction as the steering direction as compared with the case of straight travel according to the steering amount such as the steering angle.
However, when the display range of the image to be displayed on the display device 14 is switched, the occupant may not be able to immediately understand the display image 41 due to the switching. For example, even if the display range is switched in the display device 14 fixedly arranged in the vehicle interior 3, the occupant may not immediately notice the switch. The occupant may not be able to immediately recognize the relative direction and position of the moving object behind the vehicle reflected in the switched display image 41 with respect to the vehicle just by visually recognizing the display image 41. The occupant understands the display range of the switched display image 41 by visually recognizing the entire display image 41, and based on that understanding, determines the relative direction and position of each moving object shown in the display image 41. You need to figure it out. For example, when visually recognizing the rear mirror, the occupant understands the display range of the rear mirror before visually recognizing, so that it is not necessary to understand the display range of the rear mirror again when visually recognizing. The occupant can instantly grasp the moving object behind the vehicle simply by visually recognizing the rear mirror.
Further, the plurality of moving objects shown in the image displayed on the display device 14 are displayed in a plane on the display image 41. The occupant may not be able to easily visually recognize and grasp the distant moving object that appears small on the rear side in the display image 41 in which the nearby moving object appears large.
As described above, the external monitoring device 10 is required to be improved.

FIG. 4 is an explanatory diagram of an external monitoring device 10 provided in the automobile 1 of FIG.
The vehicle exterior monitor device 10 of FIG. 4 includes a right side rear camera 11, a left side rear camera 12, a center rear camera 13, a display device 14, a memory 15, and a monitor ECU 16 to which these are connected. In addition to this, the monitor ECU 16 may be connected to, for example, a driving operation ECU 21, an automatic driving / driving support ECU 23, and a steering control ECU 24 through an in-vehicle network 25. At least a part of the right side rear camera 11, the left side rear camera 12, the center rear camera 13, the display device 14, or the memory 15 may be connected to the monitor ECU 16 through the vehicle-mounted network 25.

The driving operation ECU 21 detects the operation of the occupant for operating the traveling of the automobile 1. The driving operation ECU 21 detects, for example, the steering operation of the steering 22. The driving operation ECU 21 outputs detection information such as the steering direction and steering amount of the steering 22 to the in-vehicle network 25. The steering amount may be the steering angle.

The automatic driving / driving support ECU 23 switches the driving of the automobile 1 between automatic driving and manual driving. In the case of automatic driving, the automatic driving / driving support ECU 23 outputs automatic control information such as a steering direction and a steering amount for automatically driving the vehicle 1 to the vehicle-mounted network 25. In the case of manual driving, the automatic driving / driving support ECU 23 acquires detection information from the vehicle-mounted network 25 and outputs control information such as the steering direction and steering amount corresponding to the detected information to the vehicle-mounted network 25. The automatic driving / driving support ECU 23 outputs control information to the vehicle-mounted network 25.

The steering control ECU 24 acquires control information related to steering from the vehicle-mounted network 25 and controls the steering device of the automobile 1. The steering control ECU 24 may acquire detection information regarding steering from the vehicle-mounted network 25 and control the steering device of the automobile 1. As a result, the traveling direction of the automobile 1 can be changed. The traveling direction of the automobile 1 changes, for example, from the front-rear direction of the automobile 1 to the forward oblique direction of the automobile 1, and then returns to the front-rear direction.

The monitor ECU 16 inputs steering control information input / output from these steering-related devices, for example, through the vehicle-mounted network 25. The driving operation ECU 21, the automatic driving / driving support ECU 23, and the steering control ECU 24 can function as a steering detection unit that detects steering for changing the traveling direction of the automobile 1 from, for example, the front-rear direction of the automobile 1.

The memory 15 records data and programs used by the monitor ECU 16 for control.

The monitor ECU 16 reads and executes the program recorded in the memory 15. As a result, the monitor ECU 16 functions as a control unit of the external monitor device 10.
The monitor ECU 16 synthesizes the image captured by the right side rear camera 11 and the image captured by the left side rear camera 12 with the image captured by the center rear camera 13, and generates the captured image 31 by the combination. .. The monitor ECU 16 may generate an image captured by the center rear camera 13 as an captured image 31. The monitor ECU 16 cuts out a part of the captured image 31 to generate the display image 41. At this time, the monitor ECU 16 may cut out a part of the captured image 31 according to a traveling situation such as steering of the automobile 1. The monitor ECU 16 outputs the display image 41 to the display device 14. As a result, the display device 14 can display the captured image 31 captured by the imaging device such as the right side rear camera 11, the left side rear camera 12, and the center rear camera 13.

FIG. 5 is a flowchart of the vehicle exterior monitor processing for obtaining the display image 41 from the captured image 31 by the monitor ECU 16 of FIG.
As a generation unit, the monitor ECU 16 repeatedly executes the process of FIG. 5 during traveling including the stop of the automobile 1. As a result, the display device 14 is updated in real time with the captured image 31 that captures the rear of the current automobile 1 captured by the imaging device such as the right side rear camera 11, the left side rear camera 12, and the center rear camera 13. Can be displayed as.

In step ST1, the monitor ECU 16 acquires the captured image 31. The monitor ECU 16 acquires, for example, the images captured by the right side rear camera 11, the left side rear camera 12, and the center rear camera 13, and combines the acquired plurality of images to acquire the captured image 31.

In step ST2, the monitor ECU 16 acquires information about steering. Here, the monitor ECU 16 acquires, for example, the steering amount corresponding to the steering direction and the steering angle based on the operation of the steering 22 by the driving operation ECU 21. The monitor ECU 16 may acquire the steering direction and the steering amount from the control information in the automatic operation or the manual operation output by the automatic operation / operation support ECU 23. As a result, the monitor ECU 16 acquires information such as the steering direction and steering amount of the automobile 1 from the driving operation ECU 21 as the steering detecting unit or the automatic driving / driving support ECU 23.

In step ST3, the monitor ECU 16 determines the presence or absence of steering. The monitor ECU 16 may determine the presence or absence of steering based on whether or not the magnitude of the steering amount is, for example, equal to or less than a predetermined threshold value close to 0 degrees. When the magnitude of the steering amount is equal to or less than the threshold value, the monitor ECU 16 determines that there is no steering and proceeds to the process in step ST4. When the magnitude of the steering amount is larger than the threshold value, the monitor ECU 16 determines that there is steering and proceeds to the process in step ST6.

In step ST4 when there is no steering, the monitor ECU 16 cuts out an image portion to be used as the display image 41 from the acquired captured image 31. For example, the monitor ECU 16 cuts out a portion of a cutout range having the same left-right width from the Y0 line, with the Y0 line at the center of the vehicle body 2 of the automobile 1 in the left-right direction as the center, from the captured image 31. Here, the cutout range may be, for example, a horizontally long rectangular range that matches the display screen of the display device 14. The number of pixels in the cutout range may be the same as the number of pixels on the display screen of the display device 14.

In step ST5, the monitor ECU 16 outputs the generated display image 41 to the display device 14. As a result, the display device 14 displays a rear image centered on the Y0 position of the vehicle body 2 as shown in FIG. 3A, for example. In FIG. 3A, the lane in which the automobile 1 travels and the lanes on both sides thereof are displayed. Further, another automobile 9 traveling behind the automobile 1 is displayed in a size corresponding to the distance from the own vehicle.

In step ST6 when there is steering, the monitor ECU 16 determines the position of the captured image 31 with respect to the cutting range for cutting out the image portion used for the display image 41 from the captured image 31. At this time, the cutout range is the same as the display screen of the display device 14 in the same horizontally long rectangular range as in step ST4, but the number of pixels of the image after the composition processing described later is the number of pixels of the display screen of the display device 14. The range may be a rectangular area that is narrower than the display screen of the display device 14 so as to match.
Then, the monitor ECU 16 moves the position of the cutout range in the captured image 31 in the steering direction by an amount corresponding to the magnitude of the steering amount. Here, the monitor ECU 16 may classify the magnitude of the steering amount into a plurality of stages, and move the position of the cutting range by a predetermined movement amount for each stage. The amount of conversion of the vehicle 1 due to steering can be considered to basically correspond to the speed and the amount of steering of the vehicle 1. In this case, the monitor ECU 16 acquires the speed of the automobile 1 together with the steering information, and cuts off by the movement amount corresponding to the conversion amount according to the conversion amount obtained in advance with respect to the speed and the steering amount of the automobile 1. The position of the output range may be moved. The movement amount corresponding to these steering amounts may be recorded in the memory 15. The monitor ECU 16 may select one from a plurality of movement amounts recorded in the memory 15 corresponding to each movement amount value by using the acquired steering amount. As a result, the monitor ECU 16 moves the cutout range of the acquired captured image 31 in the steering direction according to the acquired steering amount.

In step ST7, the monitor ECU 16 cuts out an image portion of the determined cutout range from the captured image 31. The monitor ECU 16 cuts out a portion of the cutout range determined in step ST6 from the captured image 31. As a result, the monitor ECU 16 obtains a cutout image of the cutout range according to the acquired steering direction and steering amount.

In step ST8, the monitor ECU 16 divides the cut out horizontally long rectangular cut-out image into three parts, a central part, a right part, and a left part. Here, the central portion may be in a range having an even predetermined width from the Y0 position to the left and right with reference to the Y0 position which is the direction directly behind the automobile 1. The central portion may be wide enough to include the lane in which the automobile 1 is traveling. The right portion may be the rest on the right side of the central portion. The left portion may be the rest on the left side of the central portion.

In step ST9, the monitor ECU 16 executes processing for using each partial image divided into the central portion, the right portion, and the left portion for the display image 41. Here, the processing may be processing such as enlargement, reduction, trimming, or other transformation of the image.
The monitor ECU 16 does not have to process the image of the central portion. In this case, the height of the image of the central portion including the direction directly behind the automobile 1 is the same height as the display image 41 when the traveling direction of the automobile 1 is the front-rear direction of the automobile 1. FIG. As shown in the above, the height is the same as that of the display image 41 when the traveling direction of the automobile 1 is the front-rear direction of the automobile 1.
The monitor ECU 16 executes a vertical enlargement process so as to increase the height of the partial image on the steering side. The monitor ECU 16 may execute the vertical enlargement processing so as to increase the steering amount according to the magnitude of the steering amount. When another automobile 9 is imaged in the partial image on the steering side, the height of the other automobile 9 becomes high and the display area increases.
The monitor ECU 16 executes a vertical reduction process so as to reduce the height of the partial image on the opposite side of the steering. The monitor ECU 16 may execute a vertical reduction process so as to be reduced according to the magnitude of the steering amount. When another vehicle 9 is imaged in the partial image on the opposite side of the steering, the height of the other vehicle 9 is lowered and the display area is reduced.
Further, the monitor ECU 16 may execute the trimming process on the portion of the processed partial image that protrudes from the display frame by the display frame corresponding to the display screen of the display device 14.

In step ST10, the monitor ECU 16 synthesizes the partial images divided into three to generate the display image 41. When the height and width of the combined image are different from the display screen of the display device 14 and the number of pixels is different, the monitor ECU 16 makes the number of pixels of the combined image match the number of pixels of the display device 14. , The resolution of the composited image may be converted.
After that, the monitor ECU 16 outputs the generated display image 41 to the display device 14 by the process of step ST5, and displays the display image 41 on the display device 14.

FIG. 6 is an explanatory diagram of the image processed by FIG.
FIG. 6A shows the captured image 31 and the cutout range 36 of the thick dotted line set at a position corresponding to the steering amount. The center position IMC in the left-right direction of the cutout range 36 is shifted to the right from the Y0 position in the center in the left-right direction of the captured image 31.
FIG. 6B is a cutout image 37 cut out from the captured image 31 by the cutout range 36. The cut-out image 37 is an image of a portion to the right of the captured image 31. Further, the cut-out image 37 is divided into three parts, a central portion 38 having a predetermined width centered on the Y0 position, and a right portion 39 on the right side of the central portion 38 and a left portion 40 on the left side of the central portion 38.
FIG. 6C is a display image 41 after processing and synthesizing. The display image 41 of FIG. 6C is displayed on the display device 14 when the automobile 1 is steered.

In the display image 41 of FIG. 6C, the height of the right portion 39 on the steering side is processed to be higher according to the magnitude of the steering amount, and is higher than the central portion 38 and the left portion 40 on the opposite side of the steering. The height of the left portion 40 on the opposite side of the steering is processed to be lower according to the magnitude of the steering amount, and is lower than the central portion 38 and the right portion 39 on the steering side. The heights of the right portion 39 and the left portion 40 are processed and displayed so that the steering side is higher than the central portion 38 and the opposite side of the steering is lower than the central portion 38. Images having different heights on the left and right are displayed on the display device 14. As a result, the right portion 39 on the steering side and the left portion 40 on the opposite side of the steering of the cut-out image are displayed in different sizes.
Further, in the display image 41, the display width of the right portion 39 on the steering side is displayed wider than the left portion 40 on the opposite side of the steering depending on the steering direction and the steering amount. The right portion 39 on the steering side of the cutout image 37 is displayed with greater emphasis than the left portion 40 on the opposite side of the steering. When the automobile 1 turns by steering, the image inside the turn is greatly emphasized and displayed.

By displaying such an outer shape display image 41 during steering, the occupant can easily understand that the image is different from that during normal traveling without steering. Further, the occupant becomes aware of the inside of the displayed image 41 during steering by emphasizing the inside of the display image 41 from the outside and displaying the display image 41 with a difference. The occupant can easily understand the inner situation by visually recognizing the inner image portion which is enlarged and easy to see in the display image 41. The occupant is aware of which part of the rear is the image part to be confirmed without checking the remaining image part to confirm which part of the rear is the image part being confirmed. You can check the image while holding it.
Further, with respect to the central portion 38 of the display image 41, an image in the direction directly behind the automobile 1 is displayed at the same height as in the case without steering. As a result, the occupant can easily recognize and visually recognize the portion of the image in the rearward direction of the automobile 1 in the display image 41.
As a result, the occupant only visually recognizes the relative position and direction of the other moving object captured in the display image 41 during steering with respect to the vehicle, and the position and range in the image in which the image is captured. It can be easily grasped.

As described above, in the present embodiment, the generation unit that acquires the image 31 captured by one or more imaging devices and generates the display image 41 to be displayed on the display device 14 is the image captured by the one or more imaging devices. An image is cut out from 31 according to the traveling situation, and a display image 41 to be displayed on the display device 14 is generated. As a result, the display device 14 can display an image in a range that is expected to be confirmed by the occupant according to the traveling situation.
In particular, the present embodiment includes a steering detection unit that detects steering for changing the traveling direction of the automobile 1 from the front-rear direction of the automobile 1. Then, the generation unit acquires information on the steering direction and the steering amount from the steering detection unit, and cuts out a part of the acquired captured image 31 from the cutting range 36 according to the acquired information on the steering direction and the steering amount. A display image 41 is generated in which the portion on the steering side and the portion on the opposite side of the steering of the cut-out image are displayed in different sizes. As a result, the portion on the steering side is displayed on the display device 14 during steering in a size different from that on the portion on the opposite side of steering. For example, the steering-side portion of the clipped image is displayed with greater emphasis than the steering-opposite portion. As a result, the occupant can distinguish the portion on the steering side from the portion on the opposite side of the steering only by visually recognizing the display image 41. The occupant can easily grasp that the image on the steering direction side is visually confirmed and confirmed only by visually recognizing the steering side portion of the display image 41, for example. The occupant directly visually recognizes the part on the steering side to see moving objects, obstacles, road surface conditions, etc. on the steering direction side without taking the procedure of visually recognizing the entire display image 41 and visually understanding the display range. It can be grasped instantly and instantly.

Then, when the steering side portion and the steering opposite side portion of the cut-out image are displayed in different sizes, the height of the steering side portion in the display image 41 is acquired from the steering detection unit. It is preferable to generate the display image 41 so as to be higher than the portion on the opposite side of the steering according to the information of the steering direction and the steering amount. Further, in the display image 41, the height of the steering side portion may be higher than the steering opposite portion as the steering amount acquired from the steering detection unit is larger. Thereby, in the display of the display device 14, it is possible to display an image in which the left and right heights are different according to the steering, and an image in which the difference changes according to the steering amount. As a result, the occupant can distinguish the portion on the steering side from the portion on the opposite side of the steering by the difference in height between the left and right sides of the displayed image without any special consciousness.

Further, the cut-out image is divided into a central portion 38 having a predetermined width including the direction directly rearward of the automobile 1 and left and right right portions 39 and left portions 40 of the central portion 38, and the heights of the right portion 39 and the left portion 40 are increased. It is preferable to generate the display image 41 so that the portion on the steering side is higher than the central portion 38 and the portion on the opposite side of the steering is lower than the central portion 38. At this time, the height of the central portion 38 including the direction directly behind the automobile 1 may be the same as the height of the display image 41 when the traveling direction of the automobile 1 is, for example, the front-rear direction of the automobile 1. As a result, the steering side portion displayed as a higher image than the other portion of the display image 41 can be easily distinguished from the other portion. The occupant can intuitively and easily understand that the Y0 position is displayed offset from the center of the image. Further, the portion on the opposite side of the steering, which is displayed as a lower image than the other portion of the display image 41, can be easily distinguished from the other portion. Further, since these parts are displayed higher or lower than the normal display image 41 in which the traveling direction of the automobile 1 is the front-rear direction of the automobile 1, the occupant who has been confirmed by the normal image by then. Can easily grasp the portion on the steering side that is displayed higher than usual and the portion on the opposite side of the steering that is displayed lower than usual by the change in the displayed height. Crew members such as drivers are less likely to be confused by the image.

The above embodiments are examples of preferred embodiments of the present invention, but the present invention is not limited thereto, and various modifications or modifications can be made without departing from the gist of the invention.

FIG. 7 is an explanatory diagram of various modifications of the image processing of FIG.
As the generation unit, the monitor ECU 16 may generate the display image 41 of the various modified examples shown in FIG. 7 by the same or similar processing as in FIG.

FIG. 7A shows an image after processing and a cutout range 36 after processing by a thick dotted line. In the process of step ST9, the monitor ECU 16 enlarges the right portion 39 of the image in both the height direction and the width direction. Therefore, the width of the other inner vehicle 9 imaged in the right portion 39 in the processed image is wider than that in the case of FIG. 6 (C), and the width is wider than that in the case of FIG. 6 (C). ) Will be displayed on the display device 14 in a larger size than in the case of). The other automobile 9 on the inside is enlarged and displayed in an easy-to-see manner in the right portion 39 of the display image 41 displayed on the display device 14. Further, by enlarging in both the vertical direction and the horizontal direction, distortion of the displayed shape from the original shape can be suppressed. It will be easier to identify the vehicle type.

FIG. 7B shows an image after the processing and a cutout range 36 after the processing by the thick dotted line. In the three-division process of the cut-out image 37 in step ST8, the monitor ECU 16 sets the central portion 38 as a range that refers to the central position of the image instead of the Y0 position. In this case, the other automobile 9 traveling directly behind in the same lane as the automobile 1 is easily divided into a central portion 38 and a left portion 40. The other automobile 9 will be displayed on the display device 14 in a reduced state in the left portion 40. In the present embodiment, it becomes difficult to display another automobile 9 traveling directly behind in such a state of discomfort.

FIG. 7C shows an image after the processing and a cutout range 36 after the processing by the thick dotted line. In this case, the monitor ECU 16 performs a three-division process on the captured image 31 before cutting, increases the height of the right portion 39 which is the divided inner partial image, and raises the height of the left portion 40 which is the divided outer partial image. Lower the height. After that, the monitor ECU 16 cuts out the image of the cutout range 36 from the processed image and sets it as the display image 41. The monitor ECU 16 executes the processes of steps ST8 and ST9 for the captured image 31 before cutting, and executes the process of step ST7 for the image after the process. In this case, the synthesis process of FIG. 10 is unnecessary. Even in the case of such a process, the display device 14 may display the displayed image 41 in which the steering side portion is expanded vertically and emphasized from the steering opposite side portion, as in FIG. 6C. it can.

FIG. 8 is an explanatory view of a modified example of the outer shape of the display image 41 displayed on the display device 14 during steering.

In the display image 41 of FIG. 8A, the inner portion of the display image 41 on the steering side extends vertically from the remaining central portion 38 and the outer portion. For example, in the machining process of step ST9, the monitor ECU 16 may perform vertical expansion processing only on the inner portion on the steering side without reducing the outer portion on the opposite side of the steering.

In the display image 41 of FIG. 8B, the entire display image 41 is enlarged vertically toward the inside of the steering side. For example, instead of the processing process in step ST9, the monitor ECU 16 may execute the processing process so as to enlarge the entire cutout image 37 vertically toward the inside of the steering side. However, in this case, it becomes difficult to grasp the portion directly behind the automobile 1 in the processed display image 41. In the present embodiment, since the central portion 38 that is not scaled up and down appears on the outer shape of the display image 41, such a problem is unlikely to occur.

In the display image 41 of FIG. 8C, the entire inner portion of the display image 41 on the steering side is enlarged to a uniform height as a whole, and the outer portion on the opposite side of the steering is entirely uniform in height. Has been reduced to. For example, in the machining process of step ST9, the monitor ECU 16 may expand the entire inner portion to a uniform height and reduce the entire outer portion on the opposite side of the steering to a uniform height.

The display image 41 of FIG. 8D is different from that of FIG. 8C, and the outer portion on the opposite side of the steering is at the same height as the central portion 38. The entire inner portion of the display image 41 on the steering side is enlarged to a uniform height. For example, in the machining process of step ST9, the monitor ECU 16 may expand the entire inner portion to a uniform height.

FIG. 9 is a flowchart of a modified example of the vehicle exterior monitor processing of FIG.
As a generation unit, the monitor ECU 16 may repeatedly execute the process of FIG. 9 instead of FIG. 5 during traveling including the stop of the automobile 1.

In step ST21, the monitor ECU 16 determines whether or not it is the timing when the steering is started. The monitor ECU 16 compares, for example, the number of consecutive times that it is determined that steering is present in step ST3, the elapsed time since the first determination that steering is present in step ST3, and a predetermined threshold value, and if it does not exceed the threshold value, It may be determined that it is the timing when the steering is started. In this case, the monitor ECU 16 advances the process to step ST6. If it is not the timing at which the steering is started, the monitor ECU 16 advances the process to step ST23.

In step ST22, the monitor ECU 16 saves the setting of the cutting range 36, the setting of division, and the like used in this process in the memory 15.

In step ST23, the monitor ECU 16 reads out the setting of the cutting range 36, the setting of division, and the like used in the previous processing stored in the memory 15. In this case, the monitor ECU 16 executes the subsequent processes from step ST6 to step ST10 with the same settings as the previous process.

In step ST24, the monitor ECU 16 deletes the setting stored in the memory 15.

As a result, the monitor ECU 16 continues to generate the display image 41 from the captured image 31 by setting at the timing when the steering is started from the start of the steering to the end of the steering. After the steering is completed, the display image 41 is continuously generated from the captured image 31 by a normal process.
During one steering, the monitor ECU 16 continues to generate the display image 41 from the captured image 31 by setting at the timing when the steering is started. Even if the steering information changes during steering, the monitor ECU 16 maintains the cut-out range 36 at the start of steering until the steering in the steering direction is completed. As a result, the occupant can prevent the display image 41 from changing due to a change in the steering amount during steering or the like. Under the situation where the lateral acceleration during steering is acting, it is possible to make it difficult for the movement of the other following automobile 9 to be erroneously determined due to the change in the display state.
On the other hand, if the display range of the display image 41 changes during steering, for example, the display state of the moving object in the image changes due to the change in the display, and the actual relative direction of the moving object in the image changes. Even if the distances have not changed, it is easy to misunderstand them as if they were changing. Similarly, even if the actual relative direction or distance of the moving object in the image changes, it is easy to misunderstand that there is no such change.
The timing for holding the setting may be immediately after the start of steering or at a predetermined timing slightly delayed from the start of steering. Further, it may be the timing when the posture of the vehicle body 2 becomes stable after steering. The stability of the vehicle body 2 may be determined, for example, by keeping the detected value of the acceleration sensor (not shown) in a short period of time within a predetermined amplitude range.

1 ... Automobile, 2 ... Body, 10 ... Outside monitor device, 11 ... Right side rear camera, 12 ... Left side rear camera, 13 ... Center rear camera, 14 ... Display device, 15 ... Memory, 16 ... Monitor ECU, 21 ... Driving operation ECU, 23 ... Automatic operation / driving support ECU, 31 ... Captured image, 36 ... Cutout range, 37 ... Cutout image, 38 ... Central part, 39 ... Right part, 40 ... Left part, 41 ... Display image

Claims (9)

One or more imaging devices that image the rear of the vehicle,
A display device that displays the display image so that it can be visually displayed by the occupants of the vehicle.
A generation unit that cuts out an image from one or a plurality of images captured by the imaging device according to a driving situation and generates the display image to be displayed on the display device.
A steering detection unit that detects steering for changing the traveling direction of the vehicle from the front-rear direction of the vehicle, and
Have,
The generator
Steering information is acquired from the steering detection unit,
A part of the acquired captured image is cut out according to the cutout range according to the acquired steering information.
Generate a display image that displays the part on the steering side and the part on the opposite side of the steering in different sizes for the cut out image.
External monitoring device.
The generator
Generate a display image so that the part on the steering side of the clipped image is displayed larger than the part on the opposite side of the steering.
The external monitoring device according to claim 1.
The generator
In the display image, a display image is generated so that the height of the portion on the steering side is higher than the portion on the opposite side of the steering according to the steering information acquired from the steering detection unit, and the heights on the left and right are displayed on the display device. Display images with different
The external monitoring device according to claim 1 or 2.
In the display image, the height of the steering side portion becomes higher than the steering opposite portion as the steering amount acquired from the steering detection unit increases.
The external monitoring device according to claim 3.
The generator
The cut-out image is divided into a central portion including the direction directly behind the vehicle and left and right right and left portions of the central portion.
The heights of the right portion and the left portion are processed so that the steering side is higher than the central portion and the opposite side of the steering is lower than the central portion to generate a display image.
The external monitoring device according to any one of claims 1 to 4.
The height of the central portion including the rearward direction of the vehicle is the same height as the display image when the traveling direction of the vehicle is the front-rear direction of the vehicle.
The external monitoring device according to claim 5.
The generator
In the display image, a display image is generated so that the display width of the portion on the steering side is wider than the display width of the portion on the opposite side of the steering according to the steering information acquired from the steering detection unit, and the display device is used. In the displayed image, the part on the steering side is wider than the part on the opposite side of the steering.
The external monitoring device according to any one of claims 1 to 6.
The generator
The cutout range of the acquired captured image is moved in the steering direction according to the steering amount acquired from the steering detection unit.
The external monitoring device according to any one of claims 1 to 7.
The generator
Even if the steering information acquired from the steering detection unit changes during steering, the cutting range at the start of steering is maintained until the steering is completed.
The external monitoring device according to any one of claims 1 to 8.

Images