JP2021076958A - Visual recognition support device - Google Patents

Abstract

To reduce a visual load on a passenger.SOLUTION: In a monitoring system 24, a display image generation section 38 allows an inner monitor 28 to display a still image stored in an image memory 46. An object extraction section 40 searches another vehicle from captured images of cameras 30. An object information generation section 42 acquires influence degree information to determine an influence degree with respect to an own vehicle concerning the searched vehicle. An object image generation section 44 generates an object image concerning a vehicle determined to have an influence degree being equal to or more than a predetermined degree. The display image generation section 38 allows the inner monitor 28 to perform displaying by superimposing an object image on a still image. Thus, a display image of the inner monitor 28 is restricted from giving a visual stimulation to a passenger, so that the passenger can recognize an object image while reducing a visual load on the passenger.SELECTED DRAWING: Figure 1

Description

The present invention relates to a visual recognition support device that assists the driver's visual recognition.

The electronic mirror system of Patent Document 1 is provided with rear side cameras arranged in the vicinity of the left and right side mirrors to take an image of the rear of the own vehicle. Further, in the electronic mirror system, a rear side monitor is arranged at the root of each of the pillars located on both sides of the windshield, and each of the rear side monitors displays the captured image of each of the rear side cameras. To.

Japanese Unexamined Patent Publication No. 2016-22001

By the way, the monitor that displays the image of the surroundings of the vehicle is arranged at a position where the occupant (driver) can easily see it. For this reason, the image displayed on the monitor may be seen by the occupant regardless of the interest of the occupant, and the image displayed on the monitor (movement of the image, etc.) may be annoying to the occupant. ..

The present invention has been made in view of the above facts, and an object of the present invention is to provide a visual recognition support device capable of reducing the visual burden on an occupant.

The visual recognition support device according to the first aspect of the present invention includes an image pickup means for photographing the surroundings of the vehicle, a display means provided in the vehicle interior and used for displaying a photographed image of the image pickup means, and a photographed image of the image pickup means. From the information acquisition means for searching for an object predicted to affect the running of the own vehicle and acquiring the influence degree information for determining the degree of influence of the searched object on the own vehicle, and the influence degree information. For the object that exceeds the preset degree of influence, the target image generation means for generating the target image showing the object from the captured image and the still image of a predetermined pattern are displayed on the display means, and the preset is set. A display control means for controlling the target image of the object having a degree of influence exceeding the degree of influence so as to be superimposed on the still image is included.

In the first aspect, the visual recognition support device of the second aspect changes at least a part of the pattern of the still image according to the influence degree information when the display control means displays the target image.

In the visual recognition support device of the third aspect, in the first or second aspect, the target image generation means generates a road surface image showing a road marking on the captured image, and the display control means generates the target image. At the same time, the road surface image is displayed on the display means.

In the visual recognition support device of the fourth aspect, in any one of the first to third aspects, the target image generating means has an influence as the influence degree of the object becomes higher according to the influence degree information. An image in which the circumference of the object is wider than when the degree is low is generated as the target image.

In the visual recognition support device of the fifth aspect, in any one of the first to fourth aspects, the imaging means images at least the side to the rear of the own vehicle.

In the visual recognition support device of the sixth aspect, in any one of the first to fifth aspects, the information acquisition means detects the target information that makes it possible to recognize the degree of influence of the target object on the own vehicle, and the above-mentioned The display control means superimposes the target information together with the target image on the still image and displays the target information on the display means.

In the visual recognition support device of the first aspect, an image pickup means for photographing the surroundings of the vehicle is provided, and a display means used for displaying a photographed image of the image pickup means is provided in the vehicle interior.

Further, in the visual recognition support device, the information acquisition means searches for an object that is predicted to affect the running of the own vehicle on the captured image of the imaging means, and determines the degree of influence of the searched object on the own vehicle. Get the impact information of. Further, the target image generation means generates a target image showing the object from the photographed image for the object exceeding the influence degree set in advance from the influence degree information.

Here, the display control means displays a still image of a predetermined pattern on the display means. Further, when the display control means searches for an object exceeding a preset degree of influence from the captured image while the still image is displayed on the display means, and generates an object image indicating the object. , The target image is superimposed on the still image and displayed.

As a result, when an object that affects the running of the own vehicle is not searched from the captured image, a still image of a predetermined pattern is displayed on the display means, so that the image displayed on the display means does not move and is displayed. It is possible to suppress the display image of the means from becoming a visual burden. Further, when an object that affects the traveling of the own vehicle is searched, the target image is superimposed on the still image, so that it can be recognized that the object that affects the traveling of the own vehicle has been searched.

In the visual recognition support device of the second aspect, when the target image is superimposed and displayed on the display means, at least a part of the still image displayed on the display means is changed. As a result, at least a part of the still image is changed, so that the occupant can be visually changed, and it is possible to easily recognize that an object that is predicted to affect the own vehicle has been searched.

In the visual recognition support device of the third aspect, the target image generation means generates a road surface image showing the road markings on the captured image, and the road surface image is displayed overlaid on the still image together with the target image. Therefore, the relative position of the object can be easily recognized from the image on the still image displayed on the display means.

In the visual recognition support device of the fourth aspect, as the target image, as the degree of influence of the object increases, an image in which the periphery of the object is expanded as compared with the case where the degree of influence is low is generated from the captured image. As a result, the target image whose background around the target object is expanded according to the degree of influence is superimposed on the still image, so that the degree of influence of the target object on the own vehicle can be easily recognized from the target image displayed on the display means. Become.

In the visual recognition support device of the fifth aspect, the imaging means images at least the side to the rear of the own vehicle. Thereby, the display means can be used as a rearview mirror that assists the visual recognition behind the vehicle.

In the visual recognition support device of the sixth aspect, the target information indicating the degree of influence of the target object on the own vehicle is displayed on the display means. As a result, the degree of influence of the object can be grasped more accurately.

It is a block diagram which shows the schematic structure of the vehicle visual recognition device which concerns on this embodiment. It is a block diagram which shows the main part of the control part of a vehicle. It is a top view which shows the outline of the vehicle which concerns on this embodiment. (A) is a schematic view showing an inner monitor on which a captured image is displayed, and (B) is a schematic view showing an inner monitor on which a still image is displayed. (A) to (D) are schematic views showing an inner monitor on which a captured image is displayed. (A) to (D) are schematic views showing an inner monitor in which a still image is displayed corresponding to FIGS. 5 (A) to 5 (D), respectively. It is a flow chart which shows the outline of the display process on the inner monitor in the suppression mode.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
FIG. 2 shows an outline of the control unit of the vehicle 10 according to the present embodiment in a block diagram, and FIG. 3 shows the vehicle 10 in a plan view. In the drawings, the front side of the vehicle is indicated by the arrow FR, and the right side in the vehicle width direction is indicated by the arrow RH.

A by-wire system is applied to the vehicle 10 according to the present embodiment. In the vehicle 10, various sensors (not shown) detect the driving operation state of the occupant P, the operating state of the traveling mechanism, and the like, and the steering control, speed control, and the like according to the detected driving operation state and the operating state of the traveling mechanism. Braking control is performed.

As shown in FIG. 2, the vehicle 10 includes a steering ECU (Electronic Control Unit) 12A, an engine ECU 12B, a transmission ECU 12C, a brake ECU 12D, and the like.

As shown in FIG. 3, the vehicle 10 is provided with a steering wheel 14 as an operating body in the steering device, and the steering wheel 14 is driven by an occupant (driver) P seated in a seat (driver's seat) 16. Used for operation. The steering ECU 12A detects the rotation angle of the steering shaft (not shown) rotated by the steering operation of the steering wheel 14 by the occupant P, and provides an actuator or the like (not shown) so that a steering angle corresponding to the detected rotation angle can be obtained. Operate to steer the steering wheels (front wheels). Further, the engine ECU 12B controls the rotation of the engine (or the electric motor) in response to the operation of the accelerator pedal of the occupant P or the like.

Further, the transmission ECU 12C controls the transmission (automatic transmission) according to the operation of the accelerator pedal of the occupant P and the vehicle speed, and the brake ECU 12D controls the operation of the brake according to the operation of the brake pedal of the occupant P. .. As a result, the vehicle 10 is driven by the respective ECUs operating in cooperation with each other according to the driving operation of the occupant P. The drive source for traveling of the vehicle 10 is not limited to the engine, and an engine and an electric motor may be used, or an electric motor may be used instead of the engine, and the vehicle 10 is driven for traveling. The drive of the source is controlled and the vehicle runs.

The vehicle 10 is provided with an automatic driving system 18 as an automatic driving device (driving support device), and the automatic driving system 18 includes an automatic driving ECU 20. Further, the vehicle 10 is provided with a monitoring device 22A, a navigation device 22B, and the like. Each of the steering ECU 12A, the engine ECU 12B, the transmission ECU 12C, the brake ECU 12D, the monitoring device 22A, and the navigation device 22B is electrically connected to the automatic driving ECU 20.

In the vehicle 10, as a driving mode, a manual driving mode in which the vehicle travels according to the driving operation of the occupant P and an automatic driving mode in which the vehicle travels without the intervention of the occupant P can be set. In the automatic driving system 18, the automatic driving ECU 20 switches the driving mode of the vehicle 10, and the automatic driving ECU 20 controls the running of the vehicle 10 in the automatic driving mode.

The monitoring device 22A is used to capture images taken by a monitoring imaging means (camera, not shown) that captures the surroundings of the own vehicle including the front and rear of the own vehicle (vehicle 10), and other vehicles and pedestrians around the own vehicle. Etc., road markings such as white lines on the road, and changes in the relative position and relative position of the traveling object and the own vehicle (vehicle body) are detected. Further, the monitoring device 22A is provided with measuring means such as a millimeter-wave radar, and changes in the relative position and relative position between the own vehicle and the traveling object, that is, the direction of the traveling object with respect to the own vehicle, and the own vehicle and the traveling object. The distance to the vehicle, the moving speed of the traveling object, the relative speed between the own vehicle and the traveling object, and the like may be detected.

The navigation device 22B acquires map information and also acquires position information of the vehicle 10 by using a GPS (Global Positioning System) function or the like to specify the position of the vehicle 10 on the map (for example, on a display (not shown)). indicate). Further, by setting (designating) the destination, the navigation device 22B sets the travel route of the vehicle 10 to the destination based on the map information or the like, and can guide along the set travel route. There is.

The automatic operation ECU 20 switches the operation mode according to a preset timing or a request of the occupant P. Further, when the automatic operation ECU 20 is switched to the automatic operation mode, the operation of the steering ECU 12A, the engine ECU 12B, the transmission ECU 12C, the brake ECU 12D, and the like is controlled based on the information acquired from the monitoring device 22A and the navigation device 22B. At this time, the automatic driving ECU 20 controls the vehicle 10 along a preset traveling path by controlling according to the road markings detected by the monitoring device 22A, the distance to the surrounding traveling object, the relative speed, and the like. Run.

On the other hand, the vehicle 10 is provided with a monitor system 24 as a visual recognition support device. FIG. 1 shows a block diagram of the configuration of the main part of the monitor system 24.
As shown in FIGS. 1 and 2, the monitor system 24 includes a display ECU 26 including display control means, an inner monitor 28 as display means, and one or more cameras 30 as image pickup means (this implementation). In the form, three cameras (30A, 30B, 30C) are provided.

In the monitor system 24, each of the inner monitor 28 and the cameras 30 (30A to 30C) is electrically connected to the display ECU 26. Further, in the monitor system 24, a normal mode and a suppression mode are set as display modes of the inner monitor 28. The display ECU 26 switches the display mode by setting the display mode to the normal mode or the suppression mode, and controls the display on the inner monitor 28 according to the captured image captured by the camera 30.

As shown in FIG. 3, the camera 30B is arranged at the center of the rear end of the vehicle 10 in the vehicle width direction, and the camera 30B images the rear of the vehicle. The camera 30A is arranged on the left outside of the vehicle (outside of the left side door), and the camera 30A images the left side of the vehicle and the rear left of the vehicle. Further, the camera 30C is arranged on the right outside of the vehicle (outside of the left side door), and the camera 30C images the right side of the vehicle and the rear right of the vehicle. The cameras 30A to 30C may also be used as the monitoring imaging means of the monitoring device 22A. Further, the cameras 30A and 30C may be provided together with the door mirror, and the cameras 30A and 30C may be provided in the vehicle 10 instead of the door mirror.

The inner monitor 28 is arranged on the front side of the vehicle of the occupant P seated on the seat 16 on the instrument panel 10A at the front of the passenger compartment. The inner monitor 28 may be arranged at a position in the vehicle interior where the occupant P can be easily visually recognized and does not obstruct the occupant P's view outside the vehicle. The inner monitor 28 may be arranged not only on the front side of the seat 16 of the instrument panel 10A but also in the middle portion in the vehicle width direction of the instrument panel 10A, and the inner monitor 28 is not limited to the instrument panel 10A and may be arranged in the front window. It may be arranged at the upper end of the glass and mounted on the ceiling surface. Further, the inner monitor 28 may be divided into three parts and arranged in each of the left side in the vehicle width direction, the central portion and the right side in the vehicle width direction in the vehicle interior.

The inner monitor 28 includes display units 32A, 32B, and 32C whose image display surfaces are rectangular and elongated, and the display units 32A to 32C include LCD (Liquid Crystal Display) or EL (Electro Luminescence). A thin display medium such as a display can be applied. In the inner monitor 28, display units 32A to 32C are arranged in order along the vehicle width direction.

The display unit 32B is longer than the display units 32A and 32C, the longitudinal direction of the display unit 32B is the vehicle width direction, and the image display surface is directed to the rear side of the vehicle. The display unit 32A is arranged on the left side of the display unit 32B in the vehicle width direction and is connected to the display unit 32B. Has been done. Further, the display unit 32C is arranged on the right side of the display unit 32B in the vehicle width direction and is connected to the display unit 32B. Is aimed at.

As a result, in the inner monitor 28, the image display surfaces of the display units 32A to 32C are directed toward the occupant P seated in the seat 16. As the inner monitor 28, a flexible display or the like may be used, and the display units 32A to 32C may be integrated.

As shown in FIG. 1, the display ECU 26 of the monitor system 24 is formed with an image acquisition unit 34, an image processing unit 36, a display image generation unit 38, and the like. The display ECU 26 includes a microcomputer (not shown) to which a CPU, ROM, RAM, non-volatile memory (EPROM, etc.) and the like are connected by a bus. The display ECU 26 functions as an image acquisition unit 34, an image processing unit 36, a display image generation unit 38, and the like by executing the program while the CPU reads the program stored in the ROM or the like and expands it in the RAM.

The image capturing unit 34 captures captured images captured by the cameras 30A to 30C. The image processing unit 36 performs predetermined image processing on each of the captured images of the cameras 30A to 30C, and also performs viewpoint conversion processing. In the viewpoint conversion process, a virtual viewpoint and a virtual screen are set, a captured image is projected on the virtual screen, and each of the captured images is converted into an image viewed from the same viewpoint (virtual viewpoint).

Here, in the image processing unit 36, a virtual screen corresponding to the arrangement shape of the display units 32A to 32C in the inner monitor 28 is set as a virtual screen, and the viewpoint (eyes) of the occupant P with respect to the inner monitor 28 as a virtual viewpoint with respect to the virtual screen. Position) applies. The image processing unit 36 performs scaling processing on the captured images so that the same subject on different captured images has the same size. In addition, the left and right sides of each captured image projected on the virtual screen in the viewpoint conversion process are reversed.

In the monitor system 24, a normal mode and a suppression mode are provided as display modes of the inner monitor 28. 4 (A) and 4 (B) show an outline of the display image of the inner monitor 28, FIG. 4 (A) corresponds to the normal mode, and FIG. 4 (B) corresponds to the suppression mode. Has been done.

When the display mode is set to the normal mode, the display ECU 26 outputs the captured image to which the viewpoint conversion process has been performed by the display image generation unit 38 to the inner monitors 28 (display units 32A to 32C). As a result, as shown in FIG. 4A, the inner monitor 28 displays a photographed image of the rear of the vehicle on the display unit 32B, and displays a photographed image of the left side of the vehicle (diagonally to the left rear side) on the display unit 32A. , The captured image on the right side of the vehicle (diagonally to the right and rear side) is displayed on the display unit 32C. At this time, the inner monitor 28 displays road markings 50 such as a white solid line (white line) and a white broken line behind the vehicle, and other vehicles 52 traveling behind the vehicle.

On the other hand, as shown in FIG. 1, the display ECU 26 of the monitor system 24 includes a target extraction unit 40 as a search means and an extraction means, a target information generation unit 42 that constitutes an information acquisition unit together with the target extraction unit 40, and a target image. The target image generation unit 44 as a generation means is formed. Further, the display ECU 26 is provided with an image memory 46 as a storage unit, and the image memory 46 uses a ROM, a non-volatile memory (EPROM, or the like) or the like. In the display ECU 26, the CPU executes a predetermined program to function as the target extraction unit 40, the target information generation unit 42, and the target image generation unit 44.

The target extraction unit 40 searches for a predetermined object from the captured images of the cameras 30 (30A to 30C) (photographed images image-processed by the image processing unit 36). Further, when the target object is detected by the search, the target extraction unit 40 extracts an image of the detected target object. In the present embodiment, another vehicle 52 traveling around the own vehicle (rear or side) is targeted, and when the target extraction unit 40 detects the other vehicle 52 by searching the captured image, it detects it. The image of the other vehicle 52 is extracted.

Further, the target extraction unit 40 extracts an image of the road marking 50 such as a center line such as a solid white line, a solid yellow line or a white broken line on the road surface, and a lane boundary line from the captured image. The target extraction unit 40 may extract images of a road sign standing on the road surface, a guide rail laid on the road side, a soundproof wall, and the like together with the road marking 50.

The target information generation unit 42 generates influence degree information for determining the degree of influence on the own vehicle for the vehicle 52 detected by the target extraction unit 40. The influence degree information is generated by using the time change of the position (running position) of the other vehicle 52 on the captured image, the time change of the size of the image of the other vehicle 52, and the like. The influence degree information includes target information such as the vehicle type, the direction with respect to the own vehicle, the distance to the own vehicle, the relative speed with respect to the own vehicle, and the traveling speed for the other vehicle 52, and the influence degree information includes the other vehicle 52. Can include various information for predicting (determining) the degree of influence on the running of the own vehicle.

The monitor system 24 may generate influence degree information about another vehicle 52 by using a measuring means such as a millimeter wave radar. In this case, the measuring means may also be used as the monitoring device 22A, the monitoring device 22A may generate the influence degree information, and the display ECU 26 may acquire the influence degree information generated by the monitoring device 22A.

The target image generation unit 44 determines the degree of influence of the vehicle 52 on the own vehicle based on the degree of influence information about the other vehicle 52, and determines that the degree of influence on the running of the own vehicle (high degree of influence) is determined. The vehicle 52 is specified as a target vehicle (vehicle of interest). One or more pieces of information included in the influence degree information can be used to identify the target vehicle.

For example, when using information indicating the distance to the own vehicle, the target vehicle is the vehicle 52 whose distance to the own vehicle is a predetermined distance or less (eg, several hundred meters or less). When using the information on the relative speed with respect to the own vehicle, the target vehicle is the vehicle 52 having a relative speed equal to or higher than a predetermined speed (for example, several tens of km / h or more). Information on the vehicle type of the vehicle 52 and information on the speed of the own vehicle or the vehicle 52 may be combined in the determination of the target vehicle. For example, the reference value for the degree of influence may be different between a passenger car such as an ordinary car and a bus or truck (large freight car) whose body is larger than that of the passenger car. The reference value for the degree of influence may be different depending on the speed.

The target image generation unit 44 generates a target image showing the target vehicle for the target vehicle determined to have a high degree of influence. As the target image, an image of the vehicle 52 (target vehicle) extracted from the captured image can be used. Further, the target image generated by the target image generation unit 44 can include an image (background, etc.) around the target vehicle according to the influence degree information, and the target image generation unit 44 has a case where the influence degree is small. A target image in which the circumference of the target vehicle is widened as the degree of influence becomes larger than that of the target vehicle is generated.

When the target vehicle is specified, the target image generation unit 44 reads the image of the road marking 50 extracted by the target extraction unit 40, and superimposes the road surface image showing the road marking 50 and the target image showing the target vehicle. Generates a target display image (which may be combined).

In addition, the target image generation unit 44 generates target information for the target vehicle so that the occupant P can recognize the degree of influence of the target vehicle. For example, the speed information (running speed) included in the influence degree information can be used as the target information. The target information is superimposed on the target display image according to the degree of influence of the target vehicle. As a result, the target image and the road surface image can be displayed on the inner monitor 28, and the target information can be displayed on the inner monitor 28 by superimposing the target image and the road surface image.

On the other hand, a preset still image is stored in the image memory 46. An image with a pattern that can prevent the occupant from paying attention is applied to the still image. Further, the image memory 46 stores one or a plurality of still images in which at least a part of the still image is changed.

Images of various patterns such as various graphic arrangements, mosaic patterns and wood grain patterns (wood grain patterns) can be applied to the still images stored in the image memory 46. Further, the still image has a pattern in which the visual stimulus is suppressed (given) to the occupant P when displayed on the inner monitor 28, and the hue (hue, color tone) capable of suppressing the visual stimulus is suppressed. ), Brightness (gradation) and saturation (vividness) are applied. As the still image, a pattern image having a low lightness is preferable to a pattern image having a high lightness, and as a still image, a so-called sober image having a low saturation is preferable to a vivid image having a high saturation, and the still image is used. Is preferable to an image having a weak color tone rather than an image having a strong color tone. For example, as a still image, an image having a calm pattern with a light tone is preferable.

As such a still image, an instrument panel 10A provided with an inner monitor 28 or an image of a pattern matching a design in the vehicle interior can be applied. For example, an instrument panel 10A or a wood grain part in the vehicle interior can be applied. If used, an image with a wood grain pattern can be applied.

Further, the still image in which at least a part of the still image is changed has the same pattern as the still image, and in the state of being displayed on the inner monitor 28, the figures and patterns forming the pattern are displayed in the vertical direction as a whole. An image moved in either the horizontal direction or the diagonal direction, an image in which each of the figures forming the pattern is inverted in either the vertical direction, the horizontal direction, or the diagonal direction, and the color tone, gradation, etc. are changed. Images that are close to each other, such as images, can be applied.

In the present embodiment, a plurality of images having similar patterns to each other are used as a still image and a plurality of still images in which at least a part of the still image is changed, and a plurality of still images having this relationship are stored in the image memory 46. There is. As shown in FIG. 4B, one of the plurality of still images stored in the image memory 46 is displayed on the inner monitor 28 as the still image 54A, and the remaining still image stored in the image memory 46 is the still image 54A. It is used as a still image 54B in which at least a part of the image is changed. In the still image 54A shown in FIG. 4B, triangular figures are arranged vertically and horizontally, and the still image 54B with respect to the still image 54A is, for example, an image in which each of the triangular figures is inverted upside down. Will be done.

In the still images 54A and 54B, a group of still images having similar patterns to each other are stored in the image memory 46, and the occupant P prefers to select from the group of still images stored in the image memory 46 in advance. You may use it afterwards. Further, a plurality of groups of still images may be stored in the image memory 46, and the still images of any group may be selected in advance by the occupant P as he / she likes.

In the display ECU 26, when the display mode is set to the suppression mode, the display image generation unit 38 outputs one of the still images stored in the image memory 46 to the inner monitor 28 as the still image 54A. Further, in the display ECU 26, when the target vehicle is extracted and the target display image is generated, the display of the inner monitor 28 is changed to the still image 54B, and the target display image is superimposed on the still image 54B and displayed as an inner. The monitor 28 covers the still image 54B of the portion corresponding to the target vehicle and the road marking 50.

Next, as an operation of the present embodiment, control of the display image of the inner monitor 28 in the monitor system 24 will be described.
In the vehicle 10, by setting the driving mode to the manual driving mode, steering control, speed control, braking control, etc. are executed according to the driving operation of the occupant P, and the vehicle 10 travels according to the driving operation of the occupant P. Will be done. Further, in the vehicle 10, when the automatic driving mode is set as the driving mode, the automatic driving system 18 executes steering control, speed control, braking control, and the like of the vehicle 10 without the driving operation of the occupant P. As a result, the vehicle 10 is driven toward a preset destination.

On the other hand, the vehicle 10 is provided with a monitor system 24. The monitor system 24 starts operation when an ignition switch (not shown) is turned on, and ends operation when the ignition switch is turned off. The monitor system 24 may be operated prior to the start of traveling of the vehicle 10 and may be terminated when the traveling of the vehicle 10 is completed, and may be operated in accordance with the boarding and disembarking of the occupant P. In the following, the distance to the own vehicle is applied to identify the target vehicle, the speed difference between the own vehicle and the target vehicle is applied to generate the target image, and the speed information of the target vehicle is applied to the target information regarding the target vehicle. I will explain.

The monitor system 24 is provided with a normal mode and a suppression mode as display modes of the inner monitor 28, and the display ECU 26 of the monitor system 24 sets (switches) the display mode to the normal mode or the suppression mode at a preset timing. .. 5 (A) to 5 (D) show changes in the display of the inner monitor 28 in the normal mode as the vehicle 10 travels, and FIGS. 6 (A) to 6 (D) show the changes in the display of the inner monitor 28. The display of the inner monitor 28 in the suppression mode corresponding to 5 (A) to 5 (D) is shown.

When the operation mode of the vehicle 10 is set to the automatic operation mode, the display ECU 26 sets the display mode to the suppression mode (switches from the normal mode to the suppression mode). Further, even in the manual driving mode, the display ECU 26 sets the display mode to the suppression mode according to the instruction of the occupant P while the vehicle 10 is traveling on an automobile-only road such as an expressway. Further, the display ECU 26 displays when the operation mode of the vehicle 10 is set to the manual operation mode, when the operation mode of the vehicle 10 is set from the automatic operation mode to the manual operation mode, or when instructed by the occupant P. Set the display mode to normal mode.

The instruction of the occupant P in the display mode may be given by a switch operation of an operation switch (not shown) (a touch operation of the occupant P when the inner monitor 28 uses a touch panel). Further, the display mode is detected by detecting the line-of-sight direction of the occupant by photographing the face or the like of the occupant P seated in the seat 16 and detecting that the line-of-sight of the occupant is directed to the image display surface of the inner monitor 28. May be set to normal mode.

In the monitor system 24, when the display mode is set to the normal mode, the display ECU 26 captures an image (video) captured by the cameras 30 (30A to 30C) and displays the captured image on the inner monitor 28. A captured image for the purpose is generated, and the generated captured image is output to the inner monitor 28. As a result, as shown in FIGS. 5A to 5D, the inner monitor 28 displays an image (video) behind the vehicle. Further, the inner monitor 28 displays the road marking 50 and the like behind the vehicle, and the other vehicle 52 behind the vehicle is displayed larger as the other vehicle 52 approaches the own vehicle. Therefore, in the monitor system 24, the inner monitor 28 functions as a rearview mirror that assists the occupant P in visually recognizing the rear of the vehicle.

Further, in the inner monitor 28, the display surface of the display unit 32A on the left side in the vehicle width direction of the display unit 32B is directed to the diagonally rear right side, and the display surface of the display unit 32C on the right side in the vehicle width direction of the display unit 32B is diagonally rearward to the left. It is turned to the side. Further, the display unit 32A of the inner monitor 28 displays the captured image of the camera 30A on the left side in the vehicle width direction, and the display unit 32C of the inner monitor 28 displays the captured image of the camera 30C on the right side in the vehicle width direction. .. As a result, the occupant P can grasp the left side and the right side of the vehicle 10 without discomfort from the captured image displayed on the inner monitor 28.

On the other hand, in the monitor system 24, when the display mode is set to the suppression mode, the display ECU 26 changes the display of the inner monitor 28 so as to suppress the visual stimulus of the occupant P. FIG. 7 shows an outline of the display process of the display ECU 26 in the suppression mode in a flow chart. The display process shown in FIG. 7 is executed when the display mode is set to the suppression mode, and ends when the display mode is set to the normal mode.

In the flowchart shown in FIG. 7, in the first step 100, the still image 54A is read from the image memory 46 and output to the inner monitor 28. In the next step 102, another vehicle 52 as an object is searched from the image captured by the camera 30, and in step 104, it is confirmed whether or not the other vehicle 52 has been searched (detected).

Here, when another vehicle 52 is detected, an affirmative determination is made in step 104, and the process proceeds to step 106. In this step 106, the influence degree information is acquired (generated) for the detected vehicle 52. In the next step 108, it is confirmed whether or not the influence degree information is equal to or more than a predetermined value (more than or equal to a preset reference value), that is, whether or not the distance between the detected vehicle 52 and the own vehicle is within the predetermined distance. To do. At this time, if the distance between the own vehicle and the vehicle 52 is large, it is determined (predicted) that the degree of influence on the own vehicle is low, and a negative determination is made in step 108.

In the normal mode, if the other vehicle 52 is not traveling on the captured image of the camera 30, the other vehicle 52 is not displayed on the inner monitor 28. Further, as shown in FIG. 5A, when the other vehicle 52 is far away from the own vehicle on the captured image, the other vehicle 52 is displayed extremely small on the inner monitor 28, and the inner monitor 28 is displayed. On the display, it is difficult to visually recognize the other vehicle 52.

Here, if the other vehicle 52 is not detected on the captured image or the other vehicle 52 is extremely far from the own vehicle, as shown in FIG. 6A, in the suppression mode, the still image is displayed on the inner monitor 28. While 54A is displayed, the still image 54A is continuously displayed.

On the other hand, in FIG. 7, if the influence degree information of the other vehicle 52 is equal to or higher than a predetermined value (greater than or equal to a preset reference value), an affirmative determination is made in step 108 and the process proceeds to step 110. That is, when the other vehicle 52 approaches the own vehicle within a predetermined distance, an affirmative determination is made in step 108 and the process proceeds to step 110.

In this step 110, the detected vehicle 52 is set as the target vehicle (vehicle of interest) 52A, and the target image 56 that makes it possible to identify the target vehicle 52A from the captured image is generated. At this time, the target image 56 is generated so that the circumference (background) of the target vehicle 52A is widened so that the speed difference from the own vehicle is large and the degree of influence is high. The target image 56 may be an image showing only the target vehicle 52A without including the surrounding image.

Further, in this step 110, target information such as a traveling speed is generated for the target vehicle 52A. At the same time, in step 112, the road surface image 58 is generated from the road marking 50 extracted from the captured image.

After that, in step 114, the image (still image 54A) displayed on the inner monitor 28 is changed to the still image 54B. The still image 54B is changed to another still image every time the influence degree information of the target vehicle 52A changes or at a preset time interval (for example, a few seconds interval). At this time, the still image 54B and the still image 54A may be alternately displayed on the inner monitor 28.

At the same time, in step 116, the target image 56 and the road surface image 58 are superimposed and displayed on the still image 54B of the inner monitor 28. Further, in this step 116, as the target information, the target information (for example, speed information 60) of the target vehicle 52A is displayed on the target image 56 (or may be on the still image 54B that does not overlap with the target image 56).

The speed information 60 is displayed on the inner monitor 28 when, for example, the target vehicle 52A approaches the own vehicle (for example, approaches within several tens of meters of the own vehicle) (see FIG. 6D). Further, the target image 56 and the road surface image 58 are displayed at positions corresponding to the captured images displayed on the inner monitor 28 in the normal mode.

As described above, while the monitor system 24 does not detect another vehicle 52 (target vehicle 52A) predicted to affect the running of the own vehicle on the captured image, the still image 54A is displayed on the inner monitor 28. Is displayed. When the target vehicle 52A is detected on the captured image, the target image 56 showing the target vehicle 52A is displayed on the inner monitor 28, and the target image 56 displayed on the inner monitor 28 is the degree of influence of the target vehicle 52A. It changes according to the information.

That is, as shown in FIGS. 5 (B) to 5 (D), in a state in which the vehicle 52 (target vehicle 52A) that is predicted to affect the own vehicle is detected in the captured image in the normal mode, The vehicle 52 is displayed on the inner monitor 28, and the closer the vehicle 52 is to the own vehicle, the larger the display.

On the other hand, as shown in FIGS. 6A and 6B, when the target vehicle 52A, which is predicted to affect the own vehicle, is detected in the captured image in the suppression mode, the inner monitor 28 The image displayed in is changed from the still image 54A to the still image 54B. Further, as shown in FIGS. 6 (B) to 6 (D), in the suppression mode, the inner monitor 28 displays the target image 56 which is superimposed on the still image 54B and shows the target vehicle 52A. Further, the inner monitor 28 displays a road surface image 58 showing the road surface marking 50.

Further, in the suppression mode, the closer the target vehicle 52A is to the own vehicle, the larger the target image 56 is displayed on the inner monitor 28. Further, in the target image 56 displayed on the inner monitor 28, the periphery of the target vehicle 52A is displayed wider as the target vehicle 52A approaches the own vehicle and the distance from the own vehicle becomes shorter. Further, when the target vehicle 52A approaches the own vehicle such as when passing near the own vehicle, the speed information 60 is displayed on the inner monitor 28.

If the target vehicle 52A cannot be searched (detected) on the captured image due to overtaking the own vehicle or the like, the still image 54A is displayed on the inner monitor 28. Further, when the own vehicle overtakes another vehicle 52, the inner monitor 28 displays a target image 56 (an image showing only the target vehicle 52A) excluding the periphery (background) of the target vehicle 52A.

As described above, in the monitor system 24, the still image 54A is displayed on the inner monitor 28 while the other vehicle 52 which is predicted to affect the own vehicle is not detected on the captured image. Therefore, it is possible to prevent the display image of the inner monitor 28 from giving a visual stimulus to the occupant P, and the occupant P does not care about the display image of the inner monitor 28. For example, when the captured image (video) of the camera 30 is displayed on the inner monitor 28, the movement on the captured image (for example, the movement of the landscape) may be noticed by the occupant P. As a result, the occupant P gazes at the inner monitor 28 more than necessary in order to focus his / her eyes on a specific part on the inner monitor 28. In the monitor system 24, the occupant P looks at the inner monitor 28 more than necessary. The gaze of the inner monitor 28 can be suppressed.

Further, in the monitor system 24, when the target vehicle 52A is detected, the still image 54B in which at least a part of the pattern of the still image 54A displayed on the inner monitor 28 is changed is displayed. As a result, the attention of the occupant P to the inner monitor 28 can be called. Further, in the monitor system 24, the still image 54B displayed on the inner monitor 28 is changed at a predetermined timing such as every predetermined time or every time the degree of influence of the target vehicle 52A changes. As a result, the attention (visual recognition) of the occupant P to the inner monitor 28 can be effectively urged.

Further, in the monitor system 24, the target image 56 showing the detected target vehicle 52A is superimposed on the still image 54B and displayed on the inner monitor 28. As a result, the occupant P can easily recognize that the target vehicle 52A exists around the own vehicle.

Further, in the monitor system 24, the higher the predicted degree of influence of the target vehicle 52A, the larger the target image 56 showing the target vehicle 52A is displayed on the inner monitor 28. As a result, the occupant P accurately recognizes that the target vehicle 52A has a high degree of influence on the own vehicle.

Further, in the monitor system 24, the speed information 60 of the target vehicle 52A is displayed on the inner monitor 28 together with the target image 56. As a result, the occupant P can more effectively recognize the degree of influence of the target vehicle 52A on the own vehicle.

Further, in the monitor system 24, the road surface image 58 showing the road marking 50 is displayed on the inner monitor 28 together with the target image 56. As a result, the occupant P can accurately recognize the relative position of the target vehicle 52A with respect to the own vehicle even if the still image 54B is displayed on the inner monitor 28 instead of the captured image, and the display of the inner monitor 28 is displayed by the occupant P. It can assist (support) visibility.

In the present embodiment described above, when the target image 56 showing the target vehicle 52A is displayed, the still image 54A is changed to the still image 54B. However, the target image or the like may be displayed by superimposing the still image on the still image without changing the still image displayed on the display means.

Further, the still image 54B displayed on the inner monitor 28 may be changed in color tone (hue) or the like according to the degree of influence of the target vehicle. In this case, the higher the degree of influence, the more vivid the still image 54B may be. As a result, the occupant P can more accurately grasp the degree of influence of the target vehicle from the change in the color tone of the still image 54B displayed on the inner monitor 28.

Further, in the present embodiment, an image extracted from the captured image of the camera 30 is used as the target image 56 showing the target vehicle 52A. However, the target vehicle in the target image may be an abstract image such as an illustration or a symbol. For example, when an illustration is used as the target vehicle on the target image, the illustration set for each vehicle type is used, the illustration corresponding to the vehicle type specified by the influence degree information (target information) is selected, and the target image is generated. May be good.

Further, the color of the target vehicle may be acquired as the influence degree information, and the illustration showing the target vehicle may be colored and displayed using the acquired color. Further, in the target image, the color of the target vehicle may be changed according to the degree of influence. For example, when the target vehicle is far away, the target vehicle is colored blue and the target vehicle is the own vehicle. The closer to, the more blue to yellow and the yellow to red may be changed (the illustration may be colored so that the higher the degree of influence, the stronger the red color).

In the present embodiment described above, the camera 30 images the rear side of the vehicle 10 from the side, and the photographed image is displayed on the inner monitor 28. However, one of the imaging means may image the rear of the vehicle 10 at a wide angle, and the display means may be composed of one image display medium.

Further, in the present embodiment, the vehicle 10 to which the by-wire system is applied has been described as an example. However, the visual recognition support device according to the present invention may be provided in a vehicle to which the by-wire system is not applied. Further, in the present embodiment, the application is applied to the vehicle 10 provided with the automatic driving system 18, but the visual recognition support device may be provided in the vehicle not provided with the automatic driving system 18. In these cases, the suppression mode may be set according to the instruction of the occupant P under a predetermined traveling environment such as while traveling on a motorway or the like.

10 ... Vehicle, 24 ... Monitor system (visual recognition support device), 26 ... Display ECU, 28 ... Inner monitor (display means), 30 (30A to 30C) ... Camera (imaging means) , 38 ... Display image generation unit, 40 ... Target extraction unit, 42 ... Target information generation unit, 44 ... Target image generation unit.

Claims (6)

An imaging means that captures the surroundings of the vehicle,
A display means provided in the vehicle interior and used for displaying a photographed image of the image pickup means,
An information acquisition means for searching an object predicted to affect the running of the own vehicle on the captured image of the imaging means and acquiring influence degree information for determining the degree of influence of the searched object on the own vehicle. When,
With respect to the object exceeding the influence degree preset from the influence degree information, the target image generation means for generating the target image showing the object from the photographed image, and the target image generation means.
A display control means for displaying a still image of a predetermined pattern on the display means and controlling the display of the target image of the object exceeding the preset degree of influence so as to be superimposed on the still image.
Visual assistance device including. The visual recognition support device according to claim 1, wherein when the display control means displays the target image, at least a part of the pattern of the still image is changed according to the influence degree information. The target image generation means generates a road surface image showing road markings on the captured image, and the target image generation means generates a road surface image.
The visual recognition support device according to claim 1 or 2, wherein the display control means displays the road surface image together with the target image on the display means. A claim that the target image generating means generates an image in which the periphery of the object is wider than when the influence is low as the influence of the object is increased according to the influence information. The visual recognition support device according to any one of claims 1 to 3. The visual recognition support device according to any one of claims 1 to 4, wherein the imaging means images at least the side to the rear of the own vehicle. The information acquisition means detects the target information that makes it possible to recognize the degree of influence of the target object on the own vehicle.
The visual recognition support device according to any one of claims 1 to 5, wherein the display control means superimposes the target information together with the target image on the still image and displays the target information on the display means.

Images