JP7164967B2 - Display control device and display control method - Google Patents

Description

The present invention relates to a display control device and a display control method.

Various techniques have been proposed for mounting a plurality of cameras on a vehicle, and for displaying a photographed image showing the surrounding area of the vehicle photographed by each camera on a display device mounted inside the vehicle.

For example, the vehicle surroundings visual recognition device proposed in Patent Literature 1 displays a mark portion in the end region of the monitor screen on the side opposite to the photographing direction of the camera. As a result, the screen of the monitor is shifted in the shooting direction of the camera, and it is possible to determine in which direction the image was shot by the camera. Further, for example, in the vehicle surroundings monitoring device proposed in Patent Document 2, an icon indicating which range of the surroundings of the vehicle the image displayed on the screen of the display device is the image is displayed below or beside the image. indicate.

JP-A-2002-46532 JP 2010-147516 A

However, in the conventional technology, it is difficult to instantly determine in which direction the captured image displayed on the screen of the display device represents the peripheral area of the vehicle.

The present invention has been made in view of the above problems, and it is possible to instantly determine at a glance whether the image displayed on the screen of the display device is a photographed image showing the surrounding area in which direction with respect to the vehicle. It is an object of the present invention to provide a display control technology that is suitable for

A display control device according to the present invention is a display control device that controls the display operation of a display device mounted on a vehicle, and shows a peripheral area of the vehicle photographed by any one of a plurality of cameras mounted on the vehicle. an image acquisition unit that acquires a captured image; and a position around the display image that corresponds to the shooting direction of the camera that captures the display image while displaying the captured image as a display image on the screen of the display device. and a display control unit for displaying an index representing the shooting direction of the camera at the shooting direction position of the camera , wherein the display control unit displays the display image at the shooting direction position of each camera. An image switching key for switching to an image is arranged, and the image switching key for switching to the same photographed image as the displayed image is made unselectable (first configuration).

Further, in the display control device having the first configuration, the display control unit can arrange an image switching key for switching the display image to the captured image of the camera at the shooting direction position of each camera. The image switching key for switching to the captured image different from the display image is displayed, but the image switching key for switching to the same captured image as the display image is not displayed on the screen ( second configuration).

Further, in the display control device having the first or second configuration, an operation information acquisition unit that acquires operation information related to the operation state of the vehicle, and a peripheral information acquisition unit that acquires peripheral information related to other objects around the vehicle. an acquisition unit, wherein the display control unit displays an index representing the object at the same position as the direction of the object with respect to the vehicle around the display image based on the operation information and the surrounding information. (Third configuration).

Further, in the display control device having the third configuration, the display control unit is configured to change the mode of the indicator representing the object based on the operation information and the position of the object (fourth configuration). It can be.

Further, in the display control device having the third configuration, the display control unit displays the object based on the operation information, the speed of the vehicle, the speed of the object, and the distance from the vehicle to the object. A configuration (fifth configuration) that changes the position where the indicator is displayed may be employed.

Further, in the display control device having the third configuration, the display control unit displays the object based on the operation information, the speed of the vehicle, the speed of the object, and the distance from the vehicle to the object. A configuration (sixth configuration) that changes the aspect of the index may be employed.

Further, the display control method according to the present invention is a display control method for controlling a display operation of a display device mounted on a vehicle, in which a peripheral area of the vehicle photographed by any one of a plurality of cameras mounted on the vehicle is displayed. displaying the captured image on the screen of the display device as a display image; and surrounding the display image corresponding to the shooting direction of the camera capturing the display image. a step of displaying an index representing a shooting direction of the camera at a shooting direction position, which is a position; and arranging an image switching key for switching the displayed image to the shot image of the camera at the shooting direction position of each camera. and a step of making the image switching key for switching to the captured image that is the same as the displayed image unselectable (seventh configuration).

According to the configuration of the present invention, it is possible to instantly determine at a glance whether the image displayed on the screen of the display device is the photographed image showing the surrounding area in which direction relative to the vehicle.

1 is a diagram showing the configuration of a display control device according to an embodiment; FIG. A diagram exemplifying the position where the camera is placed on the vehicle A diagram for explaining a technique for generating a synthetic image showing a peripheral area of a vehicle. FIG. 4 is a diagram exemplifying a screen display of a display device configured by a display control device; 4 is a flow chart showing an operation example of screen display of a display device by a display control device; Timing chart showing an operation example of screen display of a display device by a display control device FIG. 4 is a diagram exemplifying an operation state of screen display of a display device by a display control device;

Exemplary embodiments of the invention are described in detail below with reference to the drawings. In addition, this invention is not limited to the following contents.

Further, in the following description, the straight traveling direction of the vehicle, which is the direction from the driver's seat to the steering wheel, is referred to as the "forward direction (front)." The direction in which the vehicle travels straight ahead and is directed from the steering wheel to the driver's seat is defined as the "rearward direction (rear)". A direction that is perpendicular to the straight traveling direction of the vehicle and the vertical line and that extends from the right side to the left side of the driver facing forward is referred to as the "left direction." A direction that is perpendicular to the straight traveling direction of the vehicle and the vertical line and that is directed from the left side to the right side of the driver facing forward is referred to as the "right direction."

<1. Overview of Display Control Device>
FIG. 1 is a diagram showing the configuration of a display control device 1 according to an embodiment. The display control device 1 is mounted on a vehicle together with, for example, a display device 2 , a sensor section 3 , an imaging section 4 (vehicle camera), and an image processing device 5 . The display control device 1 controls the display operation of the display device 2 mounted on the vehicle. The display control device 1 acquires various kinds of information from, for example, the display device 2 , the sensor unit 3 , the imaging unit 4 and the image processing device 5 , and transmits control signals related to display operations to the display device 2 .

The display device 2 is a device that receives display images output from the display control device 1 . The display device 2 is, for example, a display device such as a navigation device mounted on a vehicle. The display device 2 includes a display section 21 and an operation section 22 . The display unit 21 is, for example, a liquid crystal display panel, and displays a display image output from the display control device 1 on the screen. The operation unit 22 includes, for example, a touch panel arranged on the surface of the display unit 21 and mechanical buttons arranged around the display unit 21, and can input information from the outside.

The sensor unit 3 has a plurality of sensors that detect various information related to the vehicle in which the sensor unit 3 is mounted. Information about the vehicle 6 may include information about the vehicle itself and information about the surroundings of the vehicle. In this embodiment, the sensor unit 3 includes, for example, a shift sensor 31 that detects the operating position of a shift lever of a vehicle transmission, a vehicle speed sensor 32 that detects the speed of the vehicle, a steering angle sensor 33 that detects the rotation angle of the steering, A clearance sensor 34 for detecting people, animals, vehicles, and other objects around the vehicle is included.

The clearance sensor 34 uses, for example, an ultrasonic sensor (sonar), an optical sensor using infrared rays, radar, or the like, and can detect people, animals, vehicles, and other objects around the vehicle. The clearance sensors 34 are embedded in a plurality of locations such as the front bumper, rear bumper, and door of the vehicle, for example. The clearance sensor 34 detects the presence, direction, and position of a person or other vehicle by transmitting a transmission wave toward the vicinity of the vehicle and receiving a reflected wave reflected by a person or another vehicle. Also, people, animals, vehicles, and other objects around the vehicle may be detected based on image recognition using images captured by the imaging unit 4 .

The photographing unit 4 is provided for the purpose of monitoring the situation around the vehicle. The photographing unit 4 includes four cameras 41 to 44, for example. FIG. 2 is a diagram illustrating positions where the cameras 41 to 44 are arranged on the vehicle 6. As shown in FIG.

A camera 41 is provided at the front end of the vehicle 6 . Therefore, the camera 41 is also called a front camera 41 . An optical axis 41a of the front camera 41 extends along the longitudinal direction of the vehicle 6 in plan view from above. A front camera 41 photographs the front direction of the vehicle 6 . A camera 43 is provided at the rear end of the vehicle 6 . Therefore, the camera 43 is also called a rear camera 43 . An optical axis 43a of the rear camera 43 extends along the longitudinal direction of the vehicle 6 in plan view from above. A rear camera 43 photographs the rear direction of the vehicle 6 . The mounting positions of the front camera 41 and the rear camera 43 are preferably at the left-right center of the vehicle 6, but may be slightly shifted in the left-right direction from the left-right center.

The camera 42 is provided on the right side door mirror 61 of the vehicle 6 . For this reason, the camera 42 is also called the right camera 42 . An optical axis 42a of the right camera 42 extends along the left-right direction of the vehicle 6 in plan view from above. The right camera 42 photographs the right direction of the vehicle 6 . The camera 44 is provided on the left side door mirror 62 of the vehicle 6 . For this reason, camera 44 is also referred to as left camera 44 . The optical axis 44a of the left camera 44 extends along the left-right direction of the vehicle 6 in plan view from above. The left camera 44 photographs the left direction of the vehicle 6 .

A fish-eye lens, for example, is used as the lens of the cameras 41-44. Since each of the cameras 41 to 44 has a horizontal angle of view θ of 180 degrees or more, the cameras 41 to 44 can photograph the entire circumference of the vehicle 6 in the horizontal direction.

The image processing device 5 is a device that processes images captured by the cameras 41-44. The image processing device 5 is provided for each vehicle equipped with a camera. In this embodiment, the image processing device 5 acquires and processes the captured image from the imaging unit 4 . The image processing device 5 can output the image generated by the image processing device 5 to the display control device 1 . The image processing device 5 includes an image generator 51 .

The image generating section 51 performs image processing on the captured image captured by the imaging section 4 . In this embodiment, the image generator 51 is configured as a hardware circuit capable of various image processing. In this embodiment, the image generator 51 generates a composite image showing the surrounding area of the vehicle 6 viewed from the virtual viewpoint based on the captured images captured by the cameras 41 to 44 mounted on the vehicle 6 .

<2. Generating Composite Image>
A method by which the image generator 51 generates a composite image showing the surrounding area of the vehicle 6 viewed from a virtual viewpoint will be described. FIG. 3 is a diagram for explaining a technique for generating a synthetic image CP showing the surrounding area of the vehicle 6. As shown in FIG.

The front camera 41, the right camera 42, the rear camera 43, and the left camera 44 capture four captured images P41 to P44 showing the front, right, rear, and left peripheral areas of the vehicle 6 at the same time. These four captured images P41 to P44 include data of the entire surroundings of the vehicle 6. FIG. The image generator 51 obtains these four captured images P41 to P44.

The image generator 51 projects the data (values of each pixel) included in these four captured images P41 to P44 onto a projection plane TS, which is a three-dimensional curved surface in a virtual three-dimensional space. The projection plane TS has, for example, a substantially hemispherical shape (bowl shape), and the center portion (bottom portion of the bowl) is determined as the position of the vehicle 6 .

On the projection plane TS, data of the photographed image is projected onto an area outside the area of the vehicle 6 . A correspondence relationship is determined in advance between the position of each pixel included in the captured images P41 to P44 and the position of each pixel on the projection plane TS. Table data indicating this correspondence relationship is stored in advance in a storage unit (not shown). The value of each pixel on the projection plane TS can be determined based on this correspondence relationship and the value of each pixel included in the captured images P41 to P44.

Next, the image generator 51 sets a virtual viewpoint VP for the three-dimensional space. A virtual viewpoint VP is defined by a viewpoint position and a line-of-sight direction. The image generating unit 51 can set a virtual viewpoint VP with an arbitrary viewpoint position and an arbitrary line-of-sight direction in a three-dimensional space. The image generator 51 cuts out, as an image, the data projected onto the area included in the field of view seen from the set virtual viewpoint VP on the projection plane TS. Thereby, the image generator 51 generates a composite image viewed from an arbitrary virtual viewpoint VP.

For example, as shown in FIG. 3, assuming a virtual viewpoint VPa in which the viewpoint position is directly above the vehicle 6 and the line-of-sight direction is directly below, a synthetic image (bird's-eye view image) CPa of bird's-eye view of the vehicle 6 and the surrounding area of the vehicle 6 is generated. can be generated.

An image 6p of the vehicle 6 shown in the composite image CPa is prepared in advance as data such as a bitmap and stored in advance. When generating the composite image, data of the image 6p of the vehicle 6 having a shape corresponding to the viewpoint position and line-of-sight direction of the virtual viewpoint VP of the composite image is read out and included in the composite image CPa.

In this manner, the image generation unit 51 can generate a synthetic image CPa with realistic presence by using the virtual three-dimensional projection surface TS.

Also, the peripheral area of the vehicle 6 is confirmed using a composite image CP showing the peripheral area of the vehicle 6, which is generated based on a plurality of captured images respectively captured by the plurality of cameras 41 to 44 mounted on the vehicle 6. can do. As a result, it is possible to check a blind spot area from the user's position, for example, an area blocked by the vehicle 6 from the user's position.

<3. Details of the display control device>
The display control device 1 includes an image acquisition unit 11, an operation information acquisition unit 12, a peripheral information acquisition unit 13, a display control unit 14, and a storage unit 15 shown in FIG.

The image acquisition unit 11 acquires from the vehicle a photographed image showing the surrounding area of the vehicle photographed by any one of the cameras 41 to 44 . In this embodiment, the number of cameras 41-44 is four. Furthermore, the image acquisition unit 11 acquires the image processed by the image processing device 5 .

The operation information acquisition unit 12 receives from the vehicle 6 operation information relating to the operation state of the vehicle. The operation information of the vehicle 6 includes, for example, the operation position (shift position) of the shift lever of the transmission of the vehicle 6, the speed of the vehicle 6 (vehicle speed), the rotation angle of the steering wheel (rudder angle), and the like. These operation information are received from the sensor unit 3 of the vehicle 6 .

The peripheral information acquisition unit 13 acquires peripheral information related to other objects around the vehicle from the vehicle 6 . The peripheral information of the vehicle 6 includes, for example, information related to people, animals, vehicles, and other objects around the vehicle 6, that is, the presence/absence, direction, and position of these objects. These peripheral information are received from the sensor unit 3 of the vehicle 6 .

The display control unit 14 controls the display operation of the display device 2 . The display control unit 14 displays, for example, an image captured by one of the cameras 41 to 44 or a synthesized image generated by the image processing device 5 at a predetermined position on the screen of the display device 2 as a display image. Further, the display control unit 14 controls the display content around the display image on the screen of the display device 2 based on the operation information acquired by the operation information acquisition unit 12 and the peripheral information acquired by the peripheral information acquisition unit 13. do. The details of the method of controlling the display operation of the display device 2 by the display control device 1 will be described later.

The storage unit 15 is, for example, a non-volatile memory such as a flash memory, and stores various kinds of information. The storage unit 15 stores, for example, a program as firmware, various data for the display control unit 14 to control the display operation of the display device 2, and the like. The storage unit 15 also stores various data necessary for the image acquisition unit 11, the operation information acquisition unit 12, and the peripheral information acquisition unit 13 to execute processing.

<4. Screen display configuration of display device>
FIG. 4 is a diagram illustrating a screen display of the display device 2 configured by the display control device 1. As shown in FIG. The screen of the display unit 21 of the display device 2 includes an image display unit 211 and an information display unit 212 shown in FIG.

The image display section 211 is arranged at a predetermined position on the screen of the display section 21, for example, at the center. That is, the display control unit 14 displays an image captured by one of the cameras 41 to 44 as a display image in the central portion of the screen of the display unit 21 . The image display unit 211 may display an overhead image (composite image) generated by the image processing device 5 . The image display unit 211 is configured by, for example, a rectangular shape in plan view.

The information display section 212 is arranged in four directions around the image display section 211 . The information display section 212 displays various information related to the vehicle 6 . Note that the screen of the display unit 21 is defined as a plan view of the vehicle 6 . As a result, information related to the front of the vehicle 6 is displayed on the upper side of the rectangular image display portion 211, information related to the rear of the vehicle 6 is displayed on the lower side, and information related to the right of the vehicle 6 is displayed on the right side. Such information is displayed, and information related to the left side of the vehicle 6 is displayed on the left side. As various types of information related to the vehicle 6, an index based on operation information and an index based on surrounding information are displayed. These indicators are composed of images such as icons and soft keys displayed on the screen of the display unit 21, which is a liquid crystal display panel.

Indicators representing various information related to the vehicle 6 are image switching keys F1, B1, L1, R1, display image icons F2, B2, L2, R2, front left clearance sensor detection icons F3, L3, front right clearance sensor detection icon F4. , R3, rear left clearance sensor detection icon B3, L4, rear right clearance sensor detection icon B4, R4, approaching vehicle detection icon F5, B5, L5, R5, approaching person detection icon F6, B6, L6, R6. In these symbols, "F" stands for front, "B" stands for rear, "L" stands for left, and "R" stands for right.

The image switching keys F1, B1, L1, and R1 are soft key icons for switching the displayed image of the image display unit 211 to the captured image of one of the cameras 41-44. The user can switch the display image of the image display unit 211 by touching and selecting an image switching key displayed on the information display unit 212 via the touch panel (operation unit 22). That is, the display control unit 14 arranges the operation unit 22 for switching the display image to the captured image of the camera at the shooting direction position, which is the position around the display image corresponding to the shooting direction of the camera. In this embodiment, the display control unit 14 arranges the operation unit 22 for switching the display image to the photographed image of the camera at the same position as the photographing direction of the camera with respect to the vehicle 6 on the four sides of the display image of the display device 2. .

Furthermore, the display control unit 14 controls whether or not the operation unit 22 can be used based on which camera captured the image to be displayed. That is, the display control unit 14 does not cause the information display unit 212 to display an image switching key for switching to the same captured image as the display image.

The display image icons F2, B2, L2, and R2 are semicircular icons that indicate in which direction the image displayed on the image display unit 211 is captured by the camera with respect to the vehicle 6 . In the information display section 212, one of the display image icons in the same direction as the shooting direction of the camera that shoots the display image on the image display section 211 is displayed. That is, the display control unit 14 displays the display image icon representing the shooting direction of the camera at the shooting direction position around the display image corresponding to the shooting direction of the camera shooting the display image. In this embodiment, when the screen of the display device 2 is a plan view of the vehicle 6, the display control unit 14 displays the display image at the same position as the shooting direction of the camera shooting the display image of the vehicle 6 in the four directions around the display image. to display a display image icon representing the shooting direction of the camera.

Front left clearance sensor detection icons F3 and L3, front right clearance sensor detection icons F4 and R3, rear left clearance sensor detection icons B3 and L4, and rear right clearance sensor detection icons B4 and R4 are the front left and front of the vehicle 6, respectively. Arc-shaped icons indicating detection distances of other objects on the right, rear left, and rear right. The arc icon includes multiple concentric arc images that are displayed based on the distance from the vehicle 6 to other detected objects.

Regarding the approach of another object to the vehicle 6 in detail, the approach of another object from the outside to the vehicle 6 is determined based on, for example, the object entering within a predetermined range from the vehicle 6 . As the predetermined range of the vehicle 6, a range such as within a radius of 10 m can be arbitrarily set.

Also, the approach of another object from the outside to the vehicle 6 is determined based on, for example, the speed of the vehicle 6, the speed of the object, and the distance from the vehicle 6 to the object. The expected arrival time of the object to the vehicle 6 is calculated based on the approach speed of the object (relative speed of the object with respect to the vehicle 6) and the distance to the object. For example, when “distance to object”/“approaching speed of object”=“estimated arrival time” is 10 seconds or less, it is determined that another object is approaching the vehicle 6 . The expected arrival time can be set to any time.

Each clearance sensor detection icon is not displayed when another object is separated from the vehicle 6 by more than a predetermined distance, or when the estimated time of arrival exceeds a predetermined time. Each clearance sensor detection icon is displayed when another object approaches the vehicle 6 within a predetermined distance or when the estimated time of arrival of the other object becomes less than a predetermined time. Additionally, each clearance sensor detection icon displays an increasing number of arc images as other objects approach the vehicle 6 .

The approaching vehicle detection icons F5, B5, L5, and R5 are icons that indicate that the vehicle 6 is approaching another vehicle. Whether the object approaching the vehicle 6 is a vehicle is detected based on image recognition using images captured by the cameras 41 to 44, for example. Each approaching vehicle detection icon is displayed when another vehicle approaches the vehicle 6 within a predetermined distance or when the expected time of arrival of the other vehicle is less than a predetermined time. Furthermore, each approaching vehicle detection icon may change its size and color as other vehicles approach the vehicle 6 .

Approaching person detection icons F<b>6 , B<b>6 , L<b>6 , and R<b>6 are icons indicating that a person approaches the vehicle 6 . Whether the object approaching the vehicle 6 is a person is detected based on image recognition using images captured by the cameras 41 to 44, for example. Each approaching person detection icon is displayed when a person approaches the vehicle 6 within a predetermined distance or when the expected arrival time of the person becomes less than a predetermined time. Furthermore, each approaching person detection icon may change its size and color as a person approaches the vehicle 6 .

In the present embodiment, the information display unit 212 is configured such that the indicator based on the operation information and the indicator based on the peripheral information are images such as icons and softkeys on the liquid crystal display panel. is not limited to For example, the information display unit 212 may be a mechanical display, buttons, or the like configured using an LED (Light Emitting Diode) or the like around the liquid crystal display panel.

Further, the information display section 212 may have an image switching key for switching the display image of the image display section 211 to a bird's-eye view image showing the surrounding area of the vehicle 6 .

<5. Operation Flow of Display Control Device>
FIG. 5 is a flow chart showing an operation example of screen display of the display device 2 by the display control device 1 . An example of the screen display operation of the display device 2 by the display control device 1 will be described with reference to the processing flow of FIG.

When the display control device 1 starts operating, it acquires a plurality of captured images showing the surrounding area of the vehicle 6 from each of the cameras 41 to 44 (step S101). The display control device 1 acquires images captured by the cameras 41 to 44 through the image acquisition unit 11 and stores them in the storage unit 15 .

Next, the operation state of the vehicle 6 is detected (step S102). The operation state of the vehicle 6 is detected based on the operation information of the vehicle 6 acquired by the operation information acquisition unit 12 . Operation information of the vehicle 6 can be obtained from the shift sensor 31, the vehicle speed sensor 32, the steering angle sensor 33, etc. of the sensor unit 3 of the vehicle 6. FIG. Thereby, the display control device 1 recognizes, for example, the traveling direction of the vehicle 6 (step S103).

Next, it is determined whether or not an image switching key for a direction other than the direction of travel is ON (step S104). If the image switching key other than the direction of travel is not ON (No in step S104), the display control device 1 causes the captured image in the direction of travel to be displayed on the screen of the display device 2 (step S105).

Subsequently, the display control device 1 displays a display image icon indicating the shooting direction of the camera at the shooting direction position, which is a position around the display image corresponding to the shooting direction of the camera shooting the display image of the display device 2. is displayed (step S106). In this embodiment, when the screen of the display device 2 is a plan view of the vehicle 6, the display control device 1 is positioned at the same position as the imaging direction of the camera that is imaging the display image of the vehicle 6 in the four directions around the display image. to display a display image icon representing the shooting direction of the camera. Subsequently, on the screen of the display device 2, the display control device 1 turns off the display of the image switching key for the direction of travel, and turns on the display of the image switching keys other than the direction of travel (step S107).

Further, in step S103, if the image switching key other than the direction of travel is ON (Yes in step S103), the display control device 1 displays the photographed image in the direction selected by the user on the screen of the display device 2. display (step S108).

Subsequently, the display control device 1 displays a display image icon indicating the shooting direction of the camera at the shooting direction position, which is a position around the display image corresponding to the shooting direction of the camera shooting the display image of the display device 2. is displayed (step S109). In this embodiment, when the screen of the display device 2 is a plan view of the vehicle 6, the display control device 1 is positioned at the same position as the imaging direction of the camera that is imaging the display image of the vehicle 6 in the four directions around the display image. to display a display image icon representing the shooting direction of the camera. Subsequently, on the screen of the display device 2, the display control device 1 turns off the display of the image switching key in the selection direction, and turns on the display of the image switching keys other than the selection direction (step S110).

Next, the display control device 1 determines whether or not there is an object approaching the vehicle 6 (step S111). Here, determination is made based on the peripheral information of the vehicle 6 acquired by the peripheral information acquisition unit 13 . The peripheral information of the vehicle 6 can be acquired by image recognition using the clearance sensor 34 of the sensor unit 3 of the vehicle 6 and the photographed image of the photographing unit 4, or the like. As a result, the display control device 1 acquires information about the presence, direction, and position of other objects approaching the vehicle 6, for example. Then, it is determined whether or not another object has approached the vehicle 6 within a predetermined distance or not, and whether or not the predicted arrival time of the other object has become less than or equal to a predetermined time.

If there is an object approaching the vehicle 6 (Yes in step S111), it is determined whether or not the traveling direction of the vehicle 6 is substantially the same as the direction in which another object approaches (step S112). If the traveling direction of the vehicle 6 and the approaching direction of another object are substantially the same (Yes in step S112), the display control device 1 displays the approaching object display, that is, the clearance sensor detection icon, with respect to the traveling direction of the vehicle 6. , the approaching vehicle detection icon, and the approaching person detection icon are turned on based on the distance to the object and the type (step S113).

If there is no object approaching the vehicle 6 (No in step S111), or if there is an object approaching the vehicle 6, but the traveling direction of the vehicle 6 is different from the direction in which another object approaches (No in step S112) ), the display control device 1 turns off the approaching object display such as the clearance sensor detection icon, the approaching vehicle detection icon, and the approaching person detection icon (step S114).

Then, returning to step S101, the display control device 1 again acquires the captured images of the cameras 41-44.

<6. Example of screen display operation by display control device>
FIG. 6 is a timing chart showing an operation example of screen display of the display device 2 by the display control device 1. FIG. Here, an example of the screen display operation of the display device 2 based on the user's vehicle operation and display operation, and the vehicle approaching from behind will be described.

As shown in FIG. 6, when the user first performs a shift operation at time t1 to set the shift position to the "D" range, the image displayed on the image display unit 211 is the image captured by the front camera 41 (on), and information is displayed. A display image icon F2 is displayed in the portion 212 . That is, the images captured by the rear camera 43, the left camera 44, and the right camera 42 are not displayed (off) in the image display section 211, and the display image icons B2, L2, and R2 are not displayed in the information display section 212. FIG. Regarding the display of the image switching keys, the image switching key F1 of the front camera 41 is not displayed (off), and the image switching keys B1, L1, and R1 of the rear camera 43, left camera 44, and right camera 42 are displayed (on). .

Next, it is assumed that the surrounding situation changes at time t2 and another vehicle approaching the vehicle 6 from behind is detected. In this case, since the shift position is in the "D" range and the traveling direction of the vehicle 6 is forward, the rear approaching vehicle detection icon B5 is not displayed.

Subsequently, when the user performs a shift operation at time t3 to set the shift position to the "R" range, the image displayed on the image display unit 211 becomes the image captured by the rear camera 43 (on), and the image displayed on the information display unit 212. An icon B2 is displayed. That is, the images captured by the front camera 41, the left camera 44, and the right camera 42 are not displayed (off) in the image display section 211, and the display image icons F2, L2, and R2 are not displayed in the information display section 212. FIG. Regarding the display of the image switching keys, the image switching key B1 of the rear camera 43 is not displayed (off), and the image switching keys F1, L1, and R1 of the front camera 41, left camera 44, and right camera 42 are displayed (on). . Then, as the traveling direction of the vehicle 6 becomes rearward, the rearward approaching vehicle detection icon B5 is displayed (on).

Subsequently, when the user performs a display operation at time t4 to switch the display image of the image display unit 211 to the captured image of the front camera 41, the displayed image of the image display unit 211 becomes the captured image of the front camera 41 (on). , the display image icon F2 is displayed on the information display section 212. FIG. That is, the images captured by the rear camera 43, the left camera 44, and the right camera 42 are not displayed (off) in the image display section 211, and the display image icons B2, L2, and R2 are not displayed in the information display section 212. FIG. Regarding the display of the image switching keys, the image switching key F1 of the front camera 41 is not displayed (off), and the image switching keys B1, L1, and R1 of the rear camera 43, left camera 44, and right camera 42 are displayed (on). . Since the shift position is in the "R" range and the traveling direction of the vehicle 6 remains rearward, the rearward approaching vehicle detection icon B5 is displayed (on).

Next, at time t5, the surrounding situation changes and the vehicle approaching the vehicle 6 behind is out of the detection range. In this case, the shift position is in the "R" range and the traveling direction of the vehicle 6 is backward, but the display of the rearward approaching vehicle detection icon B5 disappears.

<7. List of operation states of screen display by display control device>
FIG. 7 is a diagram illustrating an operation state of screen display of the display device 2 by the display control device 1. As shown in FIG. Note that "display camera" in FIG. 7 indicates in which direction the image displayed on the image display unit 211 is the photographed image of the camera. As before, the "F" in "viewing camera" stands for front, "B" stands for rear, "L" stands for left, and "R" stands for right. "D" of the "shift position" represents the drive range, "R" represents the reverse range, "N" represents the neutral range, and "P" represents the shift position of the parking range.

According to FIG. 7, for example, in "case 1", the displayed image on the image display unit 211 is "F", that is, the image captured by the front camera 41, the shift position is in the "D" range, and the vehicle speed is "0". indicates the case. At this time, the image switching key F1 is not displayed (off), and the display image icon F2 is displayed (on) in the front information area of the information display section 212, ie, the upper side of the information display section 212 in FIG. When the front left clearance sensor detection icon F3, the front right clearance sensor detection icon F4, the approaching vehicle detection icon F5, and the approaching person detection icon F6 can be displayed (on), and another object is detected in front of the vehicle 6 is displayed, and is not displayed if it is not detected.

In the rear information area of the information display portion 212 of "case 1", that is, in the lower side of the information display portion 212 in FIG. 4, the image switching key B1 is displayed and selectable (on), and the display image icon B2 is not displayed. (off). The rear left clearance sensor detection icon B3, the rear right clearance sensor detection icon B4, the approaching vehicle detection icon B5, and the approaching person detection icon B6 are not displayed (off). , is not displayed even if it is not detected.

In the left and right information areas of the information display section 212 of "case 1", that is, in the left and right sides of the information display section 212 in FIG. 4, the image switching keys L1 and R1 are displayed and selectable ( on), and the display image icons L2 and R2 are not displayed (off). The front left clearance sensor detection icon L3 and the front right clearance sensor detection icon R3 are in a displayable state (on), and are displayed when another object is detected in front of the vehicle 6, and not displayed when not detected. . The rear left clearance sensor detection icon L4, the rear right clearance sensor detection icon R4, the approaching vehicle detection icons L5 and R5, and the approaching person detection icons L6 and R6 are not displayed (off). Even if it is detected, it is not displayed even if it is not detected.

Further, for example, in “case 6”, the displayed image on the image display unit 211 is “F”, that is, the image captured by the front camera 41, the shift position is in the “R” range, the vehicle speed is “>0”, and the steering The case where the corner is '>0' is indicated. At this time, the image switching key F1 is not displayed (off), and the display image icon F2 is displayed (on) in the front information area of the information display section 212, ie, the upper side of the information display section 212 in FIG. The front left clearance sensor detection icon F3 and the front right clearance sensor detection icon F4 are in a displayable state (on) in consideration of the outer ring difference during the backward movement of the vehicle 6, and another object has been detected in front of the vehicle 6. If it is detected, it will be displayed, and if it is not detected, it will not be displayed. The approaching vehicle detection icon F5 and the approaching person detection icon F6 are not displayed (off), and are not displayed even if another object is detected in front of the vehicle 6 or not.

In the rear information area of the information display portion 212 of "case 6", that is, in the lower side of the information display portion 212 in FIG. 4, the image switching key B1 is displayed and selectable (on), and the display image icon B2 is not displayed. (off). When the rear left clearance sensor detection icon B3, the rear right clearance sensor detection icon B4, the approaching vehicle detection icon B5, and the approaching person detection icon B6 can be displayed (on), and another object is detected behind the vehicle 6 is displayed, and is not displayed if it is not detected.

The image switching keys L1 and R1 are displayed and selectable (on) in the left information area and the right information area of the information display portion 212 of "case 6", that is, in the left and right information display portions 212 of FIG. , display image icons L2 and R2 are not displayed (off). The front left clearance sensor detection icon L3 and the front right clearance sensor detection icon R3 are not displayed (off), and are not displayed even if another object is detected in front of the vehicle 6 or not. The rear left clearance sensor detection icon L4, the rear right clearance sensor detection icon R4, the approaching vehicle detection icons L5 and R5, and the approaching person detection icons L6 and R6 are in a displayable state (on), and another object is detected behind the vehicle 6. is detected, it is not displayed if it is not detected.

As described above, the display control unit 14 of the display control device 1 of the present embodiment controls the shooting direction position, which is the position around the display image corresponding to the shooting direction of the camera that is shooting the display image of the display device 2 . , an index (icon) indicating the shooting direction of the camera is displayed. Specifically, for example, when the display image on the display device 2 is the image captured by the front camera 41, the display image icon F2 is displayed in the front information area of the information display section 212, that is, the upper side of the information display section 212 in FIG. be done. The display image icons B2, L2, and R2 are not displayed in the rear information area, the left information area, and the right information area of the information display section 212, that is, in the lower, left, and right sides of the information display section 212 in FIG. As a result, it is possible to instantly determine at a glance whether the image displayed on the screen of the display device 2 is a photographed image showing the surrounding area in which direction with respect to the vehicle 6 . Then, it can be determined without moving the line of sight in which direction the display image on the screen of the display device 2 is the photographed image showing the surrounding area with respect to the vehicle 6 . Furthermore, even when the displayed image on the display device 2 is switched, it is possible to instantly determine in which direction the displayed image is taken, and there is no need to greatly move the line of sight.

For example, when the vehicle 6 is moving backward in the reverse (reverse) shift position, when the driver is checking the surroundings, that is, when the driver takes his/her eyes off the screen, the front passenger operates the screen to display the forward image. An image captured by the camera 41 may be displayed. At this time, although there is an obstacle behind the vehicle 6, an image captured by the front camera 41 with no obstacle present may be displayed. As a result, the driver may misunderstand the image displayed on the screen as the image captured by the rear camera 43 and drive backward, causing the vehicle 6 to come into contact with an obstacle present behind the vehicle 6 . The display control device 1 of the present embodiment can prevent contact due to overlooking an obstacle when the traveling direction of the vehicle 6 and the photographing direction of the display image on the screen are different. In addition, even if there is no passenger in the passenger seat, even if the screen changes due to an abnormality when the driver looks away from the screen, it is possible to prevent contact with an obstacle due to overlooking it.

In addition, the display control unit 14 arranges an image switching key (operation unit 22) for switching a display image to an image shot by the camera at the shooting direction position of the camera. Use of the image switching key (operation unit 22) is controlled based on. Specifically, for example, when the display image on the display device 2 is an image captured by the front camera 41, the image switching key F1 is displayed in the front information area of the information display section 212, that is, in the upper side of the information display section 212 in FIG. cannot be selected. Image switching keys B1, L1, and R1 are displayed and selected in the rear information area, left information area, and right information area of the information display section 212, that is, in the lower, left, and right sides of the information display section 212 in FIG. It is possible. As a result, it is possible to instantly determine at a glance whether the image displayed on the screen of the display device 2 is a photographed image showing the surrounding area in which direction with respect to the vehicle 6 . Furthermore, the user can cause the display device 2 to display a photographed image in any desired direction regardless of the traveling direction or shift position of the vehicle 6 .

Then, the display control unit 14 displays an index (icon) representing the object at the same position as the direction of the object with respect to the vehicle 6 on the four sides of the display image of the display device 2 based on the operation information of the vehicle 6 and the surrounding information. . Specifically, for example, when the shift position is in the "D" range and another object is detected in front of the vehicle 6, the front information area of the information display section 212, that is, the upper side of the information display section 212 in FIG. Any one of a front left clearance sensor detection icon F3, a front right clearance sensor detection icon F4, an approaching vehicle detection icon F5, and an approaching person detection icon F6 is displayed. As a result, even if the image displayed on the screen of the display device 2 is a photographed image showing the surrounding area in any direction with respect to the vehicle 6, it is possible to clearly grasp the direction of other objects around the vehicle 6. . Furthermore, even when the display image of the display device 2 is switched, the direction of other objects around the vehicle 6 can be clearly grasped, and there is no need to move the line of sight significantly.

The display control unit 14 also changes the mode of the index (icon) representing the object based on the operation information of the vehicle 6 and the positions of other objects around the vehicle 6 . Specifically, for example, when the shift position is in the "D" range and another object is detected in front of the vehicle 6, the front information area of the information display section 212, that is, the upper side of the information display section 212 in FIG. A front left clearance sensor detection icon F3 and a front right clearance sensor detection icon F4 are displayed. The clearance sensor detection icon increases the number of displayed arc images as other objects approach the vehicle 6 . As another example, the size and color of the approaching vehicle detection icon and approaching person detection icon are changed as another object approaches the vehicle 6 . Accordingly, even if the image displayed on the screen of the display device 2 is a photographed image showing the surrounding area in any direction with respect to the vehicle 6, the distance of the object approaching the vehicle 6 can be clearly grasped.

The display control unit 14 also changes the position where the index (icon) representing the object is displayed based on the operation information of the vehicle 6, the speed of the vehicle 6, the speed of the object, and the distance from the vehicle 6 to the object. The approach of another object from the outside to the vehicle 6 is determined, for example, by the expected arrival time calculated based on the speed of the vehicle 6, the speed of the object, and the distance from the vehicle 6 to the object. For example, when the shift position is in the "D" range and another object is detected in front of the vehicle 6, when the estimated time of arrival of the other object is equal to or less than a predetermined time, the front left clearance sensor detection icon F3, Either the front right clearance sensor detection icon F4, the approaching vehicle detection icon F5, or the approaching person detection icon F6 is displayed. This makes it possible to clearly grasp the direction of an object approaching the vehicle 6 regardless of whether the image displayed on the screen of the display device 2 is a photographed image showing the surrounding area in any direction with respect to the vehicle 6 .

Further, the display control unit 14 changes the mode of the index (icon) representing the object based on the operation information of the vehicle 6, the speed of the vehicle 6, the speed of the object, and the distance from the vehicle 6 to the object. Accordingly, even if the image displayed on the screen of the display device 2 is a photographed image showing the surrounding area in any direction with respect to the vehicle 6, the distance of the object approaching the vehicle 6 can be clearly grasped.

<8. Others>
Various modifications can be made to the various technical features disclosed in this specification without departing from the gist of the technical creation in addition to the above-described embodiments. That is, the above-described embodiments should be considered as examples in all respects and not restrictive. The technical scope of the present invention is defined by the scope of claims rather than the description of the above embodiments, and should be understood to include all modifications within the scope and meaning equivalent to the scope of claims. is.

For example, as the operation information of the vehicle 6, information related to blinker operation may be used. Thereby, the display control device 1 can recognize the traveling direction of the vehicle 6 based on the operation information of the blinker, for example.

In the above embodiment, the information display section 212 is provided around the image display section 211 in the screen of the display section 21 of the display device 2, and the keys and icons are displayed on the information display section 212. Instead of providing the information display section 212 within the screen, the information display section 212 may be provided around the screen of the display section 21 to display keys and icons. In this case, the keys and icons may be composed of light-emitting elements such as LEDs.

Further, in the above embodiments, various functions are implemented by software through arithmetic processing by the CPU according to programs, but some of these functions are implemented by electrical hardware circuits. can be Conversely, some of the functions realized by hardware circuits may be realized by software.

1 display control device 2 display device 3 sensor unit 4 imaging unit 5 image processing device 6 vehicle 11 image acquisition unit 12 operation information acquisition unit 13 peripheral information acquisition unit 14 display control unit 15 storage unit 21 display unit 22 operation unit 41, 42, 43, 44 camera 211 image display unit 212 information display unit

Claims (7)

In a display control device that controls the display operation of a display device mounted on a vehicle,
an image acquisition unit that acquires a captured image showing a peripheral area of the vehicle captured by one of a plurality of cameras mounted on the vehicle;
displaying the photographed image as a display image on the screen of the display device, and displaying the display image at a photographing direction position, which is a position around the display image corresponding to the photographing direction of the camera photographing the display image; a display control unit that displays an index representing the shooting direction ,
The display control unit arranges an image switching key for switching the display image to the captured image of the camera at the shooting direction position of each camera, and the image for switching to the same captured image as the display image. A display controller that disables the switch key . The display control unit is configured to be able to arrange an image switching key for switching the display image to the captured image of the camera at the capturing direction position of each camera, and the captured image different from the displayed image is arranged. 2. The display control device according to claim 1, wherein the image switching key for switching to , is displayed, but the image switching key for switching to the captured image that is the same as the displayed image is not displayed on the screen . an operation information acquisition unit that acquires operation information related to an operation state of the vehicle;
a peripheral information acquisition unit that acquires peripheral information related to other objects around the vehicle;
with
3. The display control unit, based on the operation information and the peripheral information, displays an index representing the object at a position around the display image in the same position as the direction of the object with respect to the vehicle. A display controller as described. 4. The display control device according to claim 3, wherein the display control unit changes the mode of the index representing the object based on the operation information and the position of the object. 4. The display control unit according to claim 3, wherein the display control unit changes the display position of the indicator representing the object based on the operation information, the speed of the vehicle, the speed of the object, and the distance from the vehicle to the object. A display controller as described. 4. The display control unit according to claim 3, wherein the display control unit changes the mode of the indicator representing the object based on the operation information, the speed of the vehicle, the speed of the object, and the distance from the vehicle to the object. Display controller. In a display control method for controlling the display operation of a display device mounted on a vehicle,
obtaining a photographed image showing a peripheral area of the vehicle photographed by one of a plurality of cameras mounted on the vehicle;
a step of displaying the captured image as a display image on the screen of the display device;
a step of displaying an indicator representing the shooting direction of the camera at a shooting direction position, which is a position around the display image corresponding to the shooting direction of the camera shooting the display image;
arranging an image switching key for switching the displayed image to the captured image of the camera at the capturing direction position of each camera;
and making the image switching key for switching to the captured image same as the displayed image unselectable .

Images