JP7353109B2 - External monitor device - Google Patents

Description

The present invention relates to an outside vehicle monitor device.

2. Description of the Related Art In vehicles such as automobiles, rear view mirrors installed in the passenger compartment, door mirrors installed on the sides of the vehicle body, etc. have been used to check the rear view.
In recent years, cameras have come to be used in vehicles instead of rear mirrors such as room mirrors and door mirrors. An image of the rear of the vehicle captured by the camera is displayed on a display device provided in the vehicle interior.
By converting a conventional mirror into a monitor, the brightness of the image can be adjusted to make it easier to see moving objects, and anti-glare treatments can be applied to weaken the headlights of moving objects behind. ,can do. It is expected that visibility will be improved by suppressing the influence of the environment outside the vehicle.
However, in the current vehicle exterior monitor device, the display device only displays a fixedly set rear image taken by, for example, a side rear camera whose imaging direction is set in advance. In this case, the image displayed on the monitor can display the same range as a conventional rear mirror or door mirror. However, as was the case with conventional rear mirrors and door mirrors, it cannot be said that the rear view area that the occupant would like to see is displayed in a way that is sufficiently visible.

Utility Model Publication No. 63-194044 JP2007-49248A

Therefore, it is conceivable to use the techniques of Patent Documents 1 and 2, for example, to switch and cut out a part of the captured image depending on the driving situation and display it on a display device.

However, even if the display range of the image displayed on the display device is switched, the occupant may not be able to fully grasp that the display range has been switched. Even if the display range is switched on a display device that is fixedly placed in the vehicle interior, the occupant may not immediately notice this. As a result, even if the displayed image is displayed on the display device, the occupant can instantly determine the relative direction and position of the moving object behind the vehicle shown in the switched display image relative to the own vehicle just by visually checking the displayed image. It doesn't necessarily mean you can recognize it. In particular, a plurality of moving objects appearing in an image displayed on a display device are displayed in a two-dimensional manner. In an image in which a nearby moving object appears large, a passenger may not be able to easily see a distant moving object that appears small behind it. The occupant must understand the display range of the switched display image by visually checking the entire display image, and must also grasp the relative direction and position of each moving object in the image. Conventionally, for example, when viewing a rear mirror, an occupant understands the display range of the rear mirror beforehand. In this case, the occupant can instantly grasp the moving object behind the vehicle by visually checking the rear mirror.

As described above, there is a need for improvements in the vehicle exterior monitoring device.

An external monitor device according to an embodiment of the present invention includes one or more imaging devices that capture an image of the rear of a vehicle, a display device that displays a display image so that it can be viewed by an occupant of the vehicle, and one or more of the a generation unit that cuts out an image from an image captured by an imaging device according to the driving situation and generates the display image to be displayed on the display device; and a steering unit that changes the traveling direction of the vehicle from the front-rear direction of the vehicle. a steering detection section for detecting, the generating section obtains steering information from the steering detection section, cuts out a part of the obtained captured image according to a cutting range according to the obtained steering information, and cuts out a part of the obtained captured image. By generating a display image so that the part on the steering side of the cut out image is displayed larger than the part on the opposite side of the steering, the part on the steering side and the part on the opposite side of the cut out image can be displayed in different sizes. Generate a display image to be displayed .

An external monitor device according to an embodiment of the present invention includes one or more imaging devices that capture an image of the rear of a vehicle, a display device that displays a display image so that it can be viewed by an occupant of the vehicle, and one or more of the a generation unit that cuts out an image from an image captured by an imaging device according to the driving situation and generates the display image to be displayed on the display device; and a steering unit that changes the traveling direction of the vehicle from the front-rear direction of the vehicle. a steering detection section for detecting, the generation section acquires steering information from the steering detection section, cuts out a part of the acquired captured image according to a cropping range according to the acquired steering information, and A display image is generated so that the height of a portion on the steering side of the display image is higher than a portion on the opposite side of the steering according to the steering information acquired from the steering detection unit, and the height of the left and right portions is displayed on the display device. By displaying images with different sizes, a display image is generated in which the steering side part and the opposite side part of the cut out image are displayed in different sizes.

Preferably, the height of the portion on the steering side in the display image becomes higher than the portion on the opposite side of the steering as the amount of steering acquired from the steering detection section increases.

An external monitor device according to an embodiment of the present invention includes one or more imaging devices that capture an image of the rear of a vehicle, a display device that displays a display image so that it can be viewed by an occupant of the vehicle, and one or more of the a generation unit that cuts out an image from an image captured by an imaging device according to the driving situation and generates the display image to be displayed on the display device; and a steering unit that changes the traveling direction of the vehicle from the front-rear direction of the vehicle. a steering detection section for detecting, the generating section obtains steering information from the steering detection section, cuts out a part of the obtained captured image according to a cutting range according to the obtained steering information, and cuts out a part of the obtained captured image. The image is divided into a central portion including the direction directly behind the vehicle, and right and left portions on the left and right sides of the central portion, and the heights of the right and left portions are set so that the steering side is higher than the central portion and the steering side is higher than the central portion. By generating a display image by processing so that the opposite side of the image is lower than the central portion, a display image is generated in which the steering side part and the opposite side part of the cut out image are displayed in different sizes. do.

Preferably, the height of the central portion including the direction directly behind the vehicle is the same height as the display image when the traveling direction of the vehicle is the front-rear direction of the vehicle.

An external monitor device according to an embodiment of the present invention includes one or more imaging devices that capture an image of the rear of a vehicle, a display device that displays a display image so that it can be viewed by an occupant of the vehicle, and one or more of the a generation unit that cuts out an image from an image captured by an imaging device according to the driving situation and generates the display image to be displayed on the display device; and a steering unit that changes the traveling direction of the vehicle from the front-rear direction of the vehicle. a steering detection section for detecting, the generation section acquires steering information from the steering detection section, cuts out a part of the acquired captured image according to a cropping range according to the acquired steering information, and Generating a display image such that a display width of a portion on the steering side of the display image is wider than a display width of a portion on the opposite side of the steering according to steering information acquired from the steering detection unit, and displaying the generated image on the display device. By making the part on the steering side of the image wider than the part on the opposite side of the steering wheel, it is possible to create a display image in which the part on the steering side and the part on the opposite side of the cropped image are displayed in different sizes. generate.

Preferably, the generation unit moves the cropping range of the acquired captured image in the direction of steering according to the amount of steering acquired from the steering detection unit.

An external monitor device according to an embodiment of the present invention includes one or more imaging devices that capture an image of the rear of a vehicle, a display device that displays a display image so that it can be viewed by an occupant of the vehicle, and one or more of the a generation unit that cuts out an image from an image captured by an imaging device according to the driving situation and generates the display image to be displayed on the display device; and a steering unit that changes the traveling direction of the vehicle from the front-rear direction of the vehicle. a steering detection section for detecting, the generating section obtains steering information from the steering detection section, cuts out a part of the obtained captured image according to a cutting range according to the obtained steering information, and cuts out a part of the obtained captured image. generates a display image that displays a portion on the steering side and a portion on the opposite side of the steering of the image in different sizes, and starts steering even if the steering information acquired from the steering detection section changes during steering. The cutting range is maintained until the steering is completed .

In the present invention, the generation unit that acquires images captured by one or more imaging devices and generates a display image to be displayed on a display device generates an image according to the driving situation from the images captured by one or more imaging devices. A display image is generated to be cut out and displayed on a display device. This allows the display device to display an image of a range that the occupant is expected to want to check depending on the driving situation.
In particular, the present invention includes a steering detection section that detects steering for changing the traveling direction of the vehicle from the front and rear directions of the vehicle. Then, the generation unit acquires the steering information from the steering detection unit, cuts out a part of the acquired captured image according to the cropping range according to the acquired steering information, and divides the steering side part of the cropped image into a part of the steering side. A display image is generated in which the opposite side is displayed in a different size. As a result, on the display device during steering, the portion on the steering side is displayed in a different size from the portion on the opposite side of the steering. For example, the portion of the cut-out image on the steering side is displayed larger than the portion on the opposite side of the steering. Thereby, the occupant can visually distinguish between the steering side part and the opposite side part by simply viewing the displayed image. For example, by simply visually recognizing the steering side portion of the displayed image, the occupant can easily understand that the image on the steering direction side is being viewed and confirmed. The crew can directly view the part on the steering side and instantly identify moving objects, obstacles, road conditions, etc. on the steering direction side, without having to visually check the entire display image and visually understand the display range. can be grasped instantly.

FIG. 1 is an explanatory diagram of a driving situation of an example of an automobile having an external monitoring device according to an embodiment of the present invention. FIG. 2 is an explanatory diagram of the arrangement of the display device of the vehicle exterior monitor device in the vehicle interior. FIG. 3 is an explanatory diagram of an example of a display image that can be displayed on the display device of the external monitor device in the driving situation of FIG. 1. FIG. 4 is an explanatory diagram of an external monitor device provided in the automobile of FIG. 1. FIG. 5 is a flowchart of an outside-vehicle monitor process performed by the monitor ECU of FIG. 3 to obtain a display image from a captured image. FIG. 6 is an explanatory diagram of an image processed according to FIG. 5. FIG. 7 is an explanatory diagram of various modifications of the image processing shown in FIG. 6. FIG. 8 is an explanatory diagram of a modified example of the external shape of the display image displayed on the display device during steering. FIG. 9 is a flowchart of a modification of the outside-vehicle monitor process of FIG. 5.

Embodiments of the present invention will be described below based on the drawings.

FIG. 1 is an explanatory diagram of an example of a driving situation of an automobile 1 having an external monitoring device 10 according to an embodiment of the present invention.
FIG. 1 shows a car 1 traveling in the center lane of a three-lane roadway that curves to the right. Behind the vehicle 1 are another vehicle 9 traveling in the same lane and another vehicle 9 traveling in the right lane on the inside of the curve of the roadway.
The automobile 1 in FIG. 1 has a vehicle body 2. The vehicle 1 shown in FIG. A vehicle interior 3 is formed in the center of the vehicle body 2 in which a passenger such as a driver rides.

Further, the automobile 1 in FIG. 1 includes an external monitor device 10. The vehicle exterior monitor device 10 displays, on the display device 14, a display image 41 based on a captured image 31 captured by an imaging device that captures an image of the rear exterior of the vehicle body 2.

A right side rear camera 11 and a left side rear camera 12 are provided on the left and right sides of the vehicle body 2 at the position of the door mirrors, instead of the door mirrors. The right side rear camera 11 and the left side rear camera 12 are installed toward the oblique rear direction of the automobile 1. Thereby, the right side rear camera 11 can image a range in the right rear direction of the vehicle body 2 from the right side surface of the vehicle body 2 to approximately the right lateral direction. The left side rear camera 12 can image a range in the left rear direction of the vehicle body 2 from the left side surface of the vehicle body 2 to approximately the left lateral direction.

A center rear camera 13 is provided in the rear center of the vehicle interior 3 of the vehicle body 2, for example. The center rear camera 13 basically only needs to be located at the Y0 position, which is the center in the left-right direction of the vehicle body 2, and may also be located in the center of the rear side of the vehicle body 2, or in the front room of the vehicle interior 3. It may be provided at the mirror position. The center rear camera 13 is installed facing directly behind the vehicle body 2. Thereby, the center rear camera 13 can image a rear range including the diagonally rearward direction of the vehicle body 2 with the center directly behind the vehicle body 2 as the center. The imaging range of the center rear camera 13 overlaps the imaging range of the right side rear camera 11 and the imaging range of the left side rear camera 12.

Note that the vehicle exterior monitor device 10 may include a 360-degree camera instead of the plurality of imaging devices such as the right side rear camera 11, the left side rear camera 12, and the center rear camera 13, or in addition to these plurality of imaging devices. . A 360-degree camera captures images all around. The 360-degree camera may be arranged, for example, on the roof of the vehicle body 2 or at the rear of the vehicle body 2. Thereby, the 360-degree camera can image the entire rear of the vehicle body 2 from the right side to the left side.

FIG. 2 is an explanatory diagram of the arrangement of the display device 14 of the vehicle exterior monitor device 10 in the vehicle interior 3. As shown in FIG.
FIG. 2 shows the front interior of the passenger compartment 3 of the automobile 1. As shown in FIG. A dashboard 5 is provided below a windshield 4 in the front interior of the vehicle interior 3.

The display device 14 of the external monitor device 10 is one liquid crystal panel. It has a single flat display surface for displaying a display image 41 on a liquid crystal panel. The display device 14 may be one in which a plurality of liquid crystal panels are arranged side by side to form a single flat display surface.
The display device 14 is embedded and fixedly provided in the dashboard 5. Thereby, the occupants of the vehicle compartment 3 can view the rear of the vehicle 1 through the display image 41 of the display device 14 disposed below while checking the front of the vehicle 1 through the windshield 4 .
The display device 14 may be provided on the dashboard 5 so that its overall position can be finely adjusted depending on, for example, the size of the occupant.

FIG. 3 is an explanatory diagram of an example of a display image 41 that can be displayed on the display device 14 of the external monitor device 10 in the driving situation shown in FIG.
FIG. 3(A) is a captured image 31 when the automobile 1 is at the position shown by the solid line in FIG. 1 and is traveling on a straight section of the roadway.
FIG. 3(B) is a captured image 31 when the automobile 1 is at the position indicated by the dotted line in FIG. 1 and is traveling on a right-curving portion of the roadway.
These captured images 31 may be a composite image of images captured by a plurality of imaging devices such as the right side rear camera 11, left side rear camera 12, and center rear camera 13, or a single image captured by the center rear camera 13 or a 360-degree camera. The image itself may be taken by one imaging device.

Normally, it is conceivable that the external monitor device 10 cuts out a certain fixed portion from the captured image 31, and cuts out a display image 41 as shown by the dotted line frame in FIGS. 3(A) and 3(B). In this case, the display image 41 is an image that is fixedly cut out from a fixed range extending left and right with the Y0 position of the vehicle body 2 as the center, regardless of the driving state of the automobile 1 or the like. Therefore, as shown in FIG. 3B, when the automobile 1 is traveling around a curve, the displayed image 41 becomes an image that is closer to the outside of the curve. Such images can be displayed over the same range as conventional rear mirrors and door mirrors, but they do not necessarily have to be visually checked by occupants, as was the case with conventional rear mirrors and door mirrors. It cannot be said that the display clearly shows the area behind the vehicle. For example, it is not possible to capture and clearly display another car 9 that is overtaking from the inside of a curve in the display image 41.

Therefore, it is conceivable to switch the images displayed on the display device 14 of the external monitor device 10 depending on, for example, the driving situation of the automobile 1. For example, it is conceivable to display on the display device 14 a display image 41 whose range is moved in the same direction as the steering direction rather than when the vehicle is traveling straight, depending on a steering amount such as a steering angle.
However, when the display range of the image displayed on the display device 14 is changed, the passenger may not be able to immediately understand the displayed image 41 due to the change. For example, even if the display range is switched on the display device 14 fixedly disposed in the vehicle interior 3, the occupant may not immediately notice the switch. The occupant may not be able to immediately recognize the relative direction and position of the rear moving object shown in the switched display image 41 with respect to the own vehicle just by viewing the display image 41. The passenger understands the display range of the switched display image 41 by visually checking the entire display image 41, and based on that understanding, determines the relative direction and position of each moving object shown in the display image 41. It is necessary to understand. For example, when viewing a rear mirror, the occupant understands the display range of the rear mirror before viewing the rear mirror, so there is no need to understand the display range of the rear mirror again when viewing the rear mirror. Passengers can instantly identify moving objects behind their vehicle simply by looking in the rear mirror.
Further, the plurality of moving objects shown in the image displayed on the display device 14 are displayed in a two-dimensional manner in the display image 41. In the display image 41 in which a nearby moving object appears large, the occupant may not be able to easily see and grasp a distant moving object that appears small behind it.
As described above, there is a need for improvement in the vehicle exterior monitoring device 10.

FIG. 4 is an explanatory diagram of the external monitor device 10 provided in the automobile 1 shown in FIG.
The external monitor device 10 of FIG. 4 includes a right side rear camera 11, a left side rear camera 12, a center rear camera 13, a display device 14, a memory 15, and a monitor ECU 16 to which these are connected. In addition to the above, a driving operation ECU 21, an automatic driving/driving support ECU 23, and a steering control ECU 24 may be connected to the monitor ECU 16 through an in-vehicle network 25. Note that at least a portion of the right side rear camera 11, the left side rear camera 12, the center rear camera 13, the display device 14, or the memory 15 may be connected to the monitor ECU 16 via the in-vehicle network 25.

The driving operation ECU 21 detects an operation by an occupant to control the driving of the automobile 1. The driving operation ECU 21 detects a steering operation of the steering wheel 22, for example. The driving operation ECU 21 outputs detection information such as the steering direction and steering amount of the steering wheel 22 to the in-vehicle network 25. The steering amount may be a steering angle.

The automatic driving/driving support ECU 23 switches the driving of the automobile 1 between automatic driving and manual driving. In the case of automatic driving, the automatic driving/driving support ECU 23 outputs automatic control information such as a steering direction and a steering amount for operating the traveling of the automobile 1 by automatic driving to the in-vehicle network 25. In the case of manual driving, the automatic driving/driving support ECU 23 acquires detection information from the in-vehicle network 25 and outputs control information such as a steering direction and a steering amount corresponding to the detected information to the in-vehicle network 25. The automatic driving/driving support ECU 23 outputs control information to the in-vehicle network 25.

The steering control ECU 24 acquires control information regarding steering from the in-vehicle network 25 and controls the steering device of the automobile 1. The steering control ECU 24 may acquire detection information regarding steering from the in-vehicle network 25 and may control the steering device of the automobile 1. Thereby, the traveling direction of the automobile 1 may change. The traveling direction of the automobile 1 changes from, for example, the longitudinal direction of the automobile 1 to the diagonal forward direction of the automobile 1, and then changes back to the longitudinal direction.

The monitor ECU 16 receives, for example, steering control information input and output from these steering-related devices through the in-vehicle network 25. The driving operation ECU 21, the automatic driving/driving support ECU 23, and the steering control ECU 24 can function as a steering detection unit that detects steering for changing the traveling direction of the automobile 1 from, for example, the front-rear direction of the automobile 1.

The memory 15 records data and programs used for control by the monitor ECU 16.

The monitor ECU 16 reads and executes a program recorded in the memory 15. Thereby, the monitor ECU 16 functions as a control section of the external monitor device 10.
The monitor ECU 16 combines the image captured by the right side rear camera 11 and the image captured by the left side rear camera 12 with the image captured by the center rear camera 13, and generates a captured image 31 by combining. . The monitor ECU 16 may generate the image captured by the central rear camera 13 as the captured image 31. The monitor ECU 16 cuts out a part of the captured image 31 to generate a display image 41. At this time, the monitor ECU 16 may cut out a part of the captured image 31 depending on the driving situation such as the steering of the automobile 1, for example. Monitor ECU 16 outputs display image 41 to display device 14 . Thereby, the captured image 31 captured by the imaging devices such as the right side rear camera 11, the left side rear camera 12, and the center rear camera 13 can be displayed on the display device 14.

FIG. 5 is a flowchart of an outside-vehicle monitor process performed by the monitor ECU 16 in FIG. 3 to obtain a display image 41 from the captured image 31.
As a generator, the monitor ECU 16 repeatedly executes the process shown in FIG. 5 while the vehicle 1 is running, including when it is stopped. As a result, the display device 14 is updated in real time with a captured image 31 of the current rear view of the vehicle 1 captured by the imaging devices such as the right side rear camera 11, the left side rear camera 12, and the center rear camera 13. It can be displayed as follows.

In step ST1, the monitor ECU 16 acquires the captured image 31. The monitor ECU 16 acquires images captured by, for example, the right side rear camera 11, the left side rear camera 12, and the center rear camera 13, and combines the acquired images to obtain a captured image 31.

In step ST2, the monitor ECU 16 acquires information regarding steering. Here, the monitor ECU 16 obtains a steering amount corresponding to a steering direction and a steering angle based on the operation of the steering wheel 22 by the driving operation ECU 21, for example. The monitor ECU 16 may acquire the steering direction and the amount of steering from the control information for automatic driving or manual driving output by the automatic driving/driving support ECU 23. Thereby, the monitor ECU 16 acquires information such as the direction of steering of the automobile 1 and the amount of steering from the driving operation ECU 21 or the automatic driving/driving support ECU 23 as a steering detecting section.

In step ST3, the monitor ECU 16 determines whether or not steering is being performed. The monitor ECU 16 may determine the presence or absence of steering, for example, based on whether the magnitude of the steering amount is less than or equal to a predetermined threshold close to 0 degrees. If the magnitude of the steering amount is less than or equal to the threshold value, the monitor ECU 16 determines that there is no steering, and advances the process to step ST4. If the magnitude of the steering amount is larger than the threshold value, the monitor ECU 16 determines that steering is present, and advances the process to step ST6.

In step ST4 when there is no steering, the monitor ECU 16 cuts out an image portion to be used as the display image 41 from the acquired captured image 31. The monitor ECU 16 cuts out from the captured image 31, for example, a cutout range having the same horizontal width from the Y0 line, centering on the Y0 line which is the center in the left-right direction of the vehicle body 2 of the automobile 1. Here, the cutting range may be, for example, a horizontally long rectangular range that matches the display screen of the display device 14. The number of pixels in the cutout range may be the same as the number of pixels on the display screen of the display device 14.

In step ST5, the monitor ECU 16 outputs the generated display image 41 to the display device 14. As a result, the display device 14 displays a rear image centered on the Y0 position of the vehicle body 2 as shown in FIG. 3(A), for example. In FIG. 3(A), the lane in which the automobile 1 is traveling and the lanes on both sides of the lane are displayed. Further, another car 9 running behind the car 1 is displayed in a size corresponding to the distance from the own car.

In step ST6 when there is steering, the monitor ECU 16 determines the position of the captured image 31 with respect to a cutting range for cutting out an image portion to be used for the display image 41 from the captured image 31. At this time, the cropping range may be a horizontally long rectangular range that is the same as the display screen of the display device 14, as in step ST4, and the number of pixels of the image after the synthesis process, which will be described later, is the same as the number of pixels on the display screen of the display device 14. To match, it may be a rectangular range that is narrower in height than the display screen of the display device 14.
Then, the monitor ECU 16 moves the position of the cutting range in the captured image 31 in the steering direction by an amount corresponding to the magnitude of the steering amount. Here, the monitor ECU 16 may classify the magnitude of the steering amount into a plurality of stages, and move the position of the cutting range by a predetermined movement amount for each stage. The amount of change in direction of the vehicle 1 due to steering can also be thought of as basically depending on the speed of the vehicle 1 and the amount of steering. In this case, the monitor ECU 16 acquires the speed of the vehicle 1 together with the steering information, and turns the vehicle 1 by the amount of movement corresponding to the amount of change of direction, based on the amount of change of direction obtained in advance for the speed of the car 1 and the amount of steering. The position of the output range may be moved. The movement amounts corresponding to these steering amounts may be recorded in the memory 15. The monitor ECU 16 may use the obtained steering amount to select one of the plurality of movement amounts recorded in the memory 15 corresponding to each movement amount value. Thereby, the monitor ECU 16 moves the cropping range of the acquired captured image 31 in the direction of steering according to the acquired steering amount.

In step ST7, the monitor ECU 16 cuts out an image portion within the determined cutting range from the captured image 31. The monitor ECU 16 cuts out a portion of the cutting range determined in step ST6 from the captured image 31. Thereby, the monitor ECU 16 obtains a cropped image of a cropping range according to the acquired steering direction and steering amount.

In step ST8, the monitor ECU 16 divides the horizontally long rectangular cutout image into three parts: a center part, a right part, and a left part. Here, the central portion may be a range having a predetermined width that is equal from the Y0 position to the left and right, with the Y0 position being the direct rear direction of the automobile 1 as a reference. The width of the central portion may be set to include the lane in which the automobile 1 is traveling. The right portion may be the remainder on the right side of the center portion. The left portion may be the remainder to the left of the center portion.

In step ST9, the monitor ECU 16 performs processing for each partial image divided into a center portion, a right portion, and a left portion for use in the display image 41. The processing processing here may be processing such as image enlargement, reduction, cropping, and other transformations.
The monitor ECU 16 does not need to process the image in the center. In this case, the height of the central portion of the image including the direction directly behind the automobile 1 is the same height as the display image 41 when the traveling direction of the automobile 1 is the longitudinal direction of the automobile 1. The height is the same as that of the display image 41 when the traveling direction of the automobile 1 is the longitudinal direction of the automobile 1 as shown in FIG.
The monitor ECU 16 executes vertical enlargement processing for the partial image on the steering side so as to increase its height. The monitor ECU 16 may perform vertical enlargement processing so that the height increases according to the magnitude of the steering amount. When another car 9 is captured in the steering side partial image, the height of the other car 9 becomes higher and the display area increases.
The monitor ECU 16 performs vertical reduction processing on the partial image on the opposite side of the steering so as to reduce its height. The monitor ECU 16 may perform vertical reduction processing so that the steering amount becomes smaller in accordance with the magnitude of the steering amount. When another car 9 is captured in the partial image on the opposite side of the steering wheel, the height of the other car 9 is lowered and the display area is reduced.
Furthermore, the monitor ECU 16 may perform trimming processing on a portion of the processed partial image that protrudes from the display frame using a display frame corresponding to the display screen of the display device 14.

In step ST10, the monitor ECU 16 generates the display image 41 by combining the three divided partial images. If the height and width of the combined image are different from those of the display screen of the display device 14 and the number of pixels is different, the monitor ECU 16 adjusts the number of pixels of the combined image to match the number of pixels of the display device 14. , the resolution of the combined image may be converted.
Thereafter, the monitor ECU 16 outputs the generated display image 41 to the display device 14 through the process of step ST5, and causes the display device 14 to display the generated display image 41.

FIG. 6 is an explanatory diagram of an image processed according to FIG. 5.
FIG. 6A shows a captured image 31 and a cutout range 36 indicated by a thick dotted line and set at a position corresponding to the amount of steering. The center position IMC in the left-right direction of the cutting range 36 is shifted to the right from the Y0 position, which is the center in the left-right direction of the captured image 31.
FIG. 6(B) shows a cutout image 37 cut out from the captured image 31 using the cutout range 36. The cutout image 37 is an image of a right-side portion of the captured image 31. Further, the cutout image 37 is divided into three parts: a central part 38 having a predetermined width centered at the Y0 position, a right part 39 to the right of the central part 38, and a left part 40 to the left of the central part 38.
FIG. 6C shows the display image 41 after processing and composition processing. The display image 41 in FIG. 6(C) is displayed on the display device 14 when the automobile 1 is being steered.

In the display image 41 of FIG. 6C, the height of the right portion 39 on the steering side is processed to be higher depending on the magnitude of the steering amount, and is higher than the center portion 38 and the left portion 40 on the opposite side of the steering. The height of the left portion 40 on the side opposite to the steering is processed to be lower depending on the magnitude of the steering amount, and is lower than the center portion 38 and the right portion 39 on the steering side. The heights of the right portion 39 and the left portion 40 are processed and displayed so that the steering side is higher than the center portion 38 and the opposite side to the steering is lower than the center portion 38. The display device 14 displays images with different heights on the left and right sides. As a result, the right portion 39 on the steering side and the left portion 40 on the opposite side of the steering of the cut out image are displayed in different sizes.
Further, in the display image 41, the display width of the right portion 39 on the steering side is displayed wider than the left portion 40 on the opposite side of the steering, depending on the steering direction and the amount of steering. The right portion 39 on the steering side of the cropped image 37 is displayed with greater emphasis than the left portion 40 on the opposite side of the steering. When the automobile 1 turns by steering, the image on the inside of the turn is displayed with great emphasis.

By displaying the display image 41 having such an external shape during steering, the occupant can easily understand that the image is different from that during normal driving without steering. In addition, the inside of the display image 41 during steering is displayed with a difference, with the inside of the display image 41 being emphasized more than the outside, so that the occupant becomes conscious of the inside. The occupant can easily understand the situation inside by viewing the enlarged inner image portion of the display image 41 that is easier to see. The occupants are not aware of which part of the image behind them they are trying to confirm, without checking the rest of the image part to confirm which part of the rear part of the image they are trying to confirm. You can check the image while holding it.
Furthermore, regarding the center portion 38 of the display image 41, an image directly behind the vehicle 1 is displayed at the same height as in the case without steering. Thereby, the occupant can easily recognize and visually recognize the portion of the image directly behind the vehicle 1 in the display image 41.
As a result, the occupant can visually confirm the relative position and direction of another moving object captured in the display image 41 during steering with respect to the own vehicle by simply visualizing the position and range in the image of the other moving object. and can be easily understood.

As described above, in this embodiment, the generation unit that acquires the image 31 captured by one or more imaging devices and generates the display image 41 to be displayed on the display device 14 is configured to generate the image captured by one or more imaging devices. 31, an image is cut out according to the driving situation, and a display image 41 to be displayed on the display device 14 is generated. Thereby, the display device 14 can display an image of a range that the occupant is expected to want to check depending on the driving situation.
In particular, the present embodiment includes a steering detection unit that detects steering for changing the traveling direction of the automobile 1 from the front-rear direction of the automobile 1. Then, the generation unit acquires the information on the steering direction and the amount of steering from the steering detection unit, and cuts out a part of the acquired captured image 31 using the cropping range 36 according to the information on the acquired steering direction and amount of steering, A display image 41 is generated in which a portion of the cut-out image on the steering side and a portion on the opposite side of the steering are displayed in different sizes. As a result, on the display device 14 during steering, the portion on the steering side is displayed in a different size from the portion on the opposite side of the steering. For example, the portion of the cut-out image on the steering side is displayed with greater emphasis than the portion on the opposite side of the steering. Thereby, the occupant can visually distinguish between the portion on the steering side and the portion on the opposite side of the steering simply by visually recognizing the display image 41. For example, by simply visually recognizing the steering side portion of the display image 41, the occupant can easily understand that he/she is viewing and confirming the image on the steering direction side. The crew member can directly view the steering side part and check the moving objects, obstacles, road surface conditions, etc. on the steering direction side without going through the steps of visually checking the entire display image 41 and visually understanding the display range. It can be grasped immediately and instantaneously.

When displaying the steering side part and the opposite side part of the cut out image in different sizes, in particular, the height of the steering side part in the display image 41 is obtained from the steering detection unit. It is preferable to generate the display image 41 so that the display image 41 is higher than the portion on the opposite side of the steering according to information on the steering direction and amount of steering. Further, it is preferable that the height of the portion on the steering side in the display image 41 becomes higher than the portion on the opposite side of the steering as the amount of steering acquired from the steering detection section increases. Thereby, in the display of the display device 14, it is possible to display an image in which the left and right heights differ depending on the steering, and an image in which the difference changes depending on the amount of steering. As a result, the occupant can visually distinguish between the steering side part and the opposite side part without being particularly conscious of the difference in height between the left and right sides of the displayed image.

Furthermore, the cut out image is divided into a central portion 38 having a predetermined width including the direction directly behind the vehicle 1, and a right portion 39 and a left portion 40 on the left and right sides of the central portion 38, and the height of the right portion 39 and left portion 40 is The display image 41 may be generated such that the portion on the steering side is higher than the central portion 38 and the portion on the opposite side of the steering is lower than the central portion 38. At this time, the height of the central portion 38 including the direction directly behind the automobile 1 may be the same height as the display image 41 when the traveling direction of the automobile 1 is the longitudinal direction of the automobile 1, for example. Thereby, the portion on the steering side that is displayed as an image higher than other portions of the display image 41 can be easily distinguished from the other portions. The occupant can easily and intuitively understand that the Y0 position is displayed shifted from the center of the image. Furthermore, the portion of the display image 41 on the opposite side of the steering wheel, which is displayed as an image lower than other portions, can be easily distinguished from the other portions. In addition, by displaying these parts higher or lower than the normal display image 41 in which the traveling direction of the vehicle 1 is the front-rear direction of the vehicle 1, the occupant who had previously confirmed the normal display image 41 can The part on the steering side that is displayed higher than usual and the part on the opposite side of steering that is displayed lower than usual can be easily understood by the change in the display height. Passengers such as the driver are less likely to be confused by the image.

Although the above embodiments are examples of preferred embodiments of the present invention, the present invention is not limited thereto, and various modifications and changes can be made without departing from the gist of the invention.

FIG. 7 is an explanatory diagram of various modifications of the image processing shown in FIG. 6.
The monitor ECU 16, as a generation unit, may generate display images 41 of various modifications shown in FIG. 7 by the same or similar processing as in FIG. 5.

FIG. 7A shows the image after processing and the cutout range 36 after processing indicated by thick dotted lines. In the process of step ST9, the monitor ECU 16 enlarges the right portion 39 of the image in both the height direction and the width direction. Therefore, in the image after processing, the other car 9 on the inside imaged in the right part 39 has a wider width than in the case of FIG. ) will be displayed on the display device 14 in a larger size than in the case of (). The other car 9 on the inside is enlarged and displayed for easy viewing in the right portion 39 of the display image 41 displayed on the display device 14. Furthermore, by enlarging the image in both the vertical and horizontal directions, distortion of the displayed shape from the original shape can be suppressed. This makes it easier to identify vehicle types.

FIG. 7B shows the image after processing and the cutout range 36 after processing indicated by thick dotted lines. In the process of dividing the cropped image 37 into three in step ST8, the monitor ECU 16 sets the center portion 38 as a range based on the center position of the image instead of the Y0 position. In this case, another vehicle 9 traveling directly behind the vehicle 1 in the same lane is likely to be separated into a center portion 38 and a left portion 40. The other car 9 will be displayed on the display device 14 in a reduced state in the left portion 40 . In the present embodiment, it becomes difficult to display other cars 9 that are running directly behind the user in such a state that gives a sense of discomfort.

FIG. 7C shows the image after processing and the cutout range 36 after processing indicated by thick dotted lines. In this case, the monitor ECU 16 processes the captured image 31 before cutting out into three parts, increases the height of the right part 39 which is the inner part of the divided image, and increases the height of the left part 40 which is the outer part of the divided image. lower the value. Thereafter, the monitor ECU 16 cuts out an image within the cutting range 36 from the processed image and sets it as a display image 41. The monitor ECU 16 executes the processes of step ST8 and step ST9 on the captured image 31 before cutting out, and executes the process of step ST7 on the image after the process. In this case, the combining process of FIG. 10 is unnecessary. Even in the case of such processing, the display device 14 can display the display image 41 in which the portion on the steering side is expanded vertically and emphasized than the portion on the opposite side of the steering, as in FIG. 6(C). can.

FIG. 8 is an explanatory diagram of a modified example of the external shape of the display image 41 displayed on the display device 14 during steering.

In the display image 41 of FIG. 8A, the inner portion on the steering side of the display image 41 is wider vertically than the remaining central portion 38 and the outer portion. For example, in the processing in step ST9, the monitor ECU 16 may perform vertical enlargement processing only on the inner portion on the steering side, without performing vertical reduction processing on the outer portion on the opposite side of the steering wheel.

In the display image 41 of FIG. 8(B), the entire display image 41 is enlarged vertically toward the inner side on the steering side. For example, instead of the processing in step ST9, the monitor ECU 16 may perform processing so as to enlarge the entire cutout image 37 vertically toward the inner side on the steering side. However, in this case, in the processed display image 41, it becomes difficult to grasp the portion directly behind the automobile 1. In this embodiment, since the central portion 38 that is not vertically scaled appears in the outer shape of the display image 41, such a problem is unlikely to occur.

In the display image 41 of FIG. 8(C), the entire inner part on the steering side of the display image 41 is enlarged to a uniform height, and the entire outer part on the opposite side of the steering is enlarged to a uniform height. has been reduced to. For example, in the processing in step ST9, the monitor ECU 16 may enlarge the entire inner portion to a uniform height and reduce the entire outer portion on the opposite side of the steering to a uniform height.

The display image 41 in FIG. 8(D) differs from FIG. 8(C) in that the outer portion on the opposite side of the steering wheel is at the same height as the central portion 38. The entire inner portion of the display image 41 on the steering side is enlarged to a uniform height. The monitor ECU 16 may enlarge the entire internal portion to a uniform height, for example, in the processing at step ST9.

FIG. 9 is a flowchart of a modification of the outside-vehicle monitor process of FIG. 5.
The monitor ECU 16, as a generating unit, may repeatedly execute the process shown in FIG. 9 instead of the process shown in FIG. 5 while the vehicle 1 is running, including when it is stopped.

In step ST21, the monitor ECU 16 determines whether or not it is the timing at which steering is started. The monitor ECU 16 compares, for example, the consecutive number of times it was determined that there was steering in step ST3, or the elapsed time since it was first determined that there was steering in step ST3, with a predetermined threshold, and if the threshold is not exceeded, It may be determined that this is the timing at which steering is started. In this case, the monitor ECU 16 advances the process to step ST6. If it is not the timing at which the steering is started, the monitor ECU 16 advances the process to step ST23.

In step ST22, the monitor ECU 16 stores in the memory 15 the settings of the cutting range 36, the division settings, etc. used in the current process.

In step ST23, the monitor ECU 16 reads the settings of the cutting range 36, the division settings, etc. used in the previous process, which are stored in the memory 15. In this case, the monitor ECU 16 executes the subsequent processes from step ST6 to step ST10 with the same settings as the previous process.

In step ST24, the monitor ECU 16 deletes the settings stored in the memory 15.

As a result, from the start of the steering until the end of the steering, the monitor ECU 16 continues to generate the display image 41 from the captured image 31 according to the settings made at the timing at which the steering was started. After the steering is completed, the display image 41 continues to be generated from the captured image 31 through normal processing.
During one steering operation, the monitor ECU 16 continues to generate the display image 41 from the captured image 31 according to the settings made at the timing when the steering was started. Even if the steering information changes during steering, the monitor ECU 16 maintains the steering range 36 at the time the steering is started until the steering in that steering direction is completed. As a result, the occupant can prevent the display image 41 from changing due to changes in the amount of steering during steering. Under a situation in which lateral acceleration is applied during steering, it is possible to prevent the movement of the following other vehicle 9 from being erroneously judged due to a change in the display state.
On the other hand, if the display range of the display image 41 changes during steering, for example, the display state of the displayed moving object changes due to the change in display, and the actual relative direction of the displayed moving object changes. Even if distances and other factors have not changed, it is easy to misinterpret them as if they have changed. Similarly, even if the actual relative direction or distance of the moving object in the image has changed, it is easy to misidentify it as if there were no changes.
Note that the timing at which the settings are held may be immediately after the start of steering, or may be at a predetermined timing slightly delayed from the start of steering. Alternatively, the timing may be when the attitude of the vehicle body 2 becomes stable after steering. The stability of the vehicle body 2 may be determined, for example, by determining that a detected value of an acceleration sensor (not shown) in a short period of time falls within a predetermined amplitude range.

DESCRIPTION OF SYMBOLS 1...Automobile, 2...Vehicle body, 10...Vehicle external monitor device, 11...Right side rear camera, 12...Left side rear camera, 13...Center rear camera, 14...Display device, 15...Memory, 16...Monitor ECU, 21... Driving operation ECU, 23...Automatic driving/driving support ECU, 31...Pictured image, 36...Cutout range, 37...Cutout image, 38...Central part, 39...Right part, 40...Left part, 41...Display image

Claims (8)

one or more imaging devices that capture images of the rear of the vehicle;
a display device that displays a display image so that it can be viewed by an occupant of the vehicle;
a generation unit that generates the display image to be displayed on the display device by cutting out an image according to the driving situation from images captured by the one or more imaging devices;
a steering detection unit that detects steering for changing the traveling direction of the vehicle from the front-rear direction of the vehicle;
has
The generation unit is
acquiring steering information from the steering detection section;
A part of the acquired captured image is cropped according to the cropping range according to the acquired steering information,
By generating a display image so that the part on the steering side of the cut out image is displayed larger than the part on the opposite side of steering, the part on the steering side and the part on the opposite side of the cut out image are made different. Generate a display image to display in size,
External monitoring device.
one or more imaging devices that capture images of the rear of the vehicle;
a display device that displays a display image so that it can be viewed by an occupant of the vehicle;
a generation unit that generates the display image to be displayed on the display device by cutting out an image according to the driving situation from images captured by the one or more imaging devices;
a steering detection unit that detects steering for changing the traveling direction of the vehicle from the front-rear direction of the vehicle;
has
The generation unit is
acquiring steering information from the steering detection section;
A part of the acquired captured image is cropped according to the cropping range according to the acquired steering information,
A display image is generated such that the height of the portion on the steering side of the display image is higher than the portion on the opposite side of the steering according to the steering information acquired from the steering detection unit, and the height of the left and right portions is displayed on the display device. generating a display image in which a portion of the cut out image on the steering side and a portion on the opposite side of the steering are displayed in different sizes by displaying images with different sizes ;
External monitoring device.
The height of the portion on the steering side in the display image becomes higher than the portion on the opposite side of the steering as the amount of steering acquired from the steering detection unit increases;
The vehicle exterior monitoring device according to claim 2 .
one or more imaging devices that capture images of the rear of the vehicle;
a display device that displays a display image so that it can be viewed by an occupant of the vehicle;
a generation unit that generates the display image to be displayed on the display device by cutting out an image according to the driving situation from images captured by the one or more imaging devices;
a steering detection unit that detects steering for changing the traveling direction of the vehicle from the front-rear direction of the vehicle;
has
The generation unit is
acquiring steering information from the steering detection section;
A part of the acquired captured image is cropped according to the cropping range according to the acquired steering information,
Divide the cut out image into a central portion including the direction directly behind the vehicle, and right and left portions on the left and right sides of the central portion,
By processing the heights of the right and left parts so that the steering side is higher than the central part and the opposite side of the steering is lower than the central part to generate a display image, the steering side part of the cut out image is generated. and a portion on the opposite side of the steering wheel at different sizes.
External monitoring device.
The height of the central portion including the direction directly behind the vehicle is the same height as the display image when the traveling direction of the vehicle is the front-rear direction of the vehicle.
The vehicle exterior monitoring device according to claim 4 .
one or more imaging devices that capture images of the rear of the vehicle;
a display device that displays a display image so that it can be viewed by an occupant of the vehicle;
a generation unit that generates the display image to be displayed on the display device by cutting out an image according to the driving situation from images captured by the one or more imaging devices;
a steering detection unit that detects steering for changing the traveling direction of the vehicle from the front-rear direction of the vehicle;
has
The generation unit is
acquiring steering information from the steering detection section;
A part of the acquired captured image is cropped according to the cropping range according to the acquired steering information,
Generating a display image such that a display width of a portion on the steering side of the display image is wider than a display width of a portion on the opposite side of steering according to steering information acquired from the steering detection unit, and displaying the display image on the display device. A display image in which the portion on the steering side and the portion on the opposite side of the cut out image are displayed in different sizes by making the portion on the steering side wider than the portion on the opposite side of the steering in the displayed image. generate,
External monitoring device.
The generation unit is
moving the cropping range of the acquired captured image in the direction of steering according to the amount of steering acquired from the steering detection section;
An external vehicle monitoring device according to any one of claims 1 to 6.
one or more imaging devices that capture images of the rear of the vehicle;
a display device that displays a display image so that it can be viewed by an occupant of the vehicle;
a generation unit that generates the display image to be displayed on the display device by cutting out an image according to the driving situation from images captured by the one or more imaging devices;
a steering detection unit that detects steering for changing the traveling direction of the vehicle from the front-rear direction of the vehicle;
has
The generation unit is
acquiring steering information from the steering detection section;
A part of the acquired captured image is cropped according to the cropping range according to the acquired steering information,
Generating a display image that displays a portion on the steering side and a portion on the opposite side of the steering of the cut out image in different sizes,
Even if the steering information acquired from the steering detection section changes during steering, the cutting range at the time of starting the steering is maintained until the steering ends.
External monitoring device.

Images