WO2015133072A1 - Vehicle peripheral image display device and method for displaying vehicle peripheral image - Google Patents

Vehicle peripheral image display device and method for displaying vehicle peripheral image Download PDF

Info

Publication number
WO2015133072A1
WO2015133072A1 PCT/JP2015/000746 JP2015000746W WO2015133072A1 WO 2015133072 A1 WO2015133072 A1 WO 2015133072A1 JP 2015000746 W JP2015000746 W JP 2015000746W WO 2015133072 A1 WO2015133072 A1 WO 2015133072A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
vehicle
display
speed
bird
Prior art date
Application number
PCT/JP2015/000746
Other languages
French (fr)
Japanese (ja)
Inventor
祥吾 大宮
純一 北尾
典史 松川
圭史 冨山
Original Assignee
株式会社デンソー
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 株式会社デンソー filed Critical 株式会社デンソー
Priority to DE112015001088.2T priority Critical patent/DE112015001088T5/en
Priority to CN201580011579.4A priority patent/CN106063258B/en
Publication of WO2015133072A1 publication Critical patent/WO2015133072A1/en

Links

Images

Classifications

    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60RVEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R1/00Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/20Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/22Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle
    • B60R1/23Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle with a predetermined field of view
    • B60R1/27Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle with a predetermined field of view providing all-round vision, e.g. using omnidirectional cameras
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00Television systems
    • H04N7/18Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60RVEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/10Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of camera system used
    • B60R2300/105Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of camera system used using multiple cameras
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60RVEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/30Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing
    • B60R2300/302Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing combining image information with GPS information or vehicle data, e.g. vehicle speed, gyro, steering angle data
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60RVEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/60Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by monitoring and displaying vehicle exterior scenes from a transformed perspective
    • B60R2300/607Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by monitoring and displaying vehicle exterior scenes from a transformed perspective from a bird's eye viewpoint
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60RVEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/70Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by an event-triggered choice to display a specific image among a selection of captured images

Definitions

  • the present disclosure relates to a technique for displaying an image obtained by photographing a peripheral area of a vehicle using an in-vehicle camera on a display screen.
  • Patent Document 1 a technique for displaying an overhead image of a peripheral region in the front-rear and left-right directions of the vehicle has been proposed (Patent Document 1) even though the vehicle-mounted cameras are not mounted on the left and right sides of the vehicle.
  • Patent Document 1 a technique for displaying an overhead image of a peripheral region in the front-rear and left-right directions of the vehicle.
  • the bird's-eye view image obtained from the front vehicle-mounted camera also includes an image of a region diagonally forward in the left and right directions of the vehicle. This diagonally forward area eventually becomes an area in the left-right direction of the vehicle as the vehicle moves forward. From this, if the bird's-eye view image is stored and an image corresponding to the region in the left-right direction of the vehicle is cut out and displayed from the stored bird's-eye view image according to the amount of movement of the vehicle, A bird's-eye view of the surrounding area including the direction can be displayed.
  • the bird's-eye view image displayed in the left-right direction of the vehicle is a past bird's-eye view image that was actually taken a little while ago. For this reason, an image cut out from a past bird's-eye view image and displayed in the left-right direction of the vehicle is called a “history image”.
  • the overhead image displayed on the screen in this way is updated every time the in-vehicle camera captures a new image, and the cycle in which the in-vehicle camera captures a new image is constant. Therefore, if the moving speed of the vehicle is known, it is possible to know how many areas move to the blind spot area in the previously captured image. Furthermore, since it is not possible to display a history image in all blind spot areas only with the last photographed image, it is only necessary to display an image photographed two times before or in the blind spot area as a history image. .
  • the inventors of the present application have found the following regarding a technique for displaying an overhead image.
  • the technology for displaying a bird's-eye view image of an area that becomes a blind spot of the in-vehicle camera using the history image may cause the display image quality to deteriorate as the moving speed of the vehicle increases, causing the driver to feel uncomfortable.
  • the part of the image taken with the in-vehicle camera must be line-of-sight converted and displayed as a history image. This is because the influence of image distortion due to line-of-sight conversion is easily noticeable.
  • This disclosure is intended to provide a technique capable of displaying a bird's-eye view of the surroundings of a vehicle without causing the driver to feel uncomfortable even when the moving speed of the vehicle is high.
  • a vehicle peripheral image display device is a vehicle peripheral image display device that displays an image of a peripheral region of a vehicle using a vehicle-mounted camera on a display screen, and the imaging obtained by the vehicle-mounted camera.
  • a photographic bird's-eye view image generation unit that generates a photographic bird's-eye view image by performing line-of-sight conversion for converting the peripheral region captured by the in-vehicle camera into an image photographed from above the vehicle.
  • a shooting bird's-eye view image storage unit for storing the shooting bird's-eye view image display unit for displaying the shooting bird's-eye view image at a position corresponding to the peripheral area shot by the vehicle-mounted camera on the display screen, and a moving speed of the vehicle.
  • a movement information acquisition unit that acquires movement information including a blind spot area that is a blind spot of the in-vehicle camera in the peripheral area of the vehicle is generated before the vehicle reaches the current position.
  • a history image that is an image of a portion of the photographed bird's-eye image captured from the blind spot area is cut out based on the movement information and displayed at a position corresponding to the blind spot area on the display screen.
  • a photographing unit, and the recording bird's-eye image storage unit is a unit that starts storing the photographing bird's-eye image when the moving speed is equal to or lower than a predetermined storage speed. The history image is displayed after the vehicle has moved a predetermined distance from the start of storage.
  • a vehicle peripheral image display method is a vehicle peripheral image display device that displays an image obtained by photographing a peripheral region of a vehicle using a vehicle-mounted camera on a display screen.
  • a shooting bird's-eye view image generating step for generating a shooting bird's-eye view image by performing line-of-sight conversion for converting the peripheral area captured by the vehicle-mounted camera into an image shot from above the vehicle with respect to the obtained shot image;
  • a shooting bird's-eye view image storing step for storing the shooting bird's-eye view image, a shooting bird's-eye view image displaying step for displaying the shooting bird's-eye view image at a position corresponding to the peripheral area shot by the vehicle-mounted camera on the display screen, and the vehicle
  • a movement information acquisition step of acquiring movement information including a movement speed of the vehicle, and a blind spot area that is a blind spot of the in-vehicle camera in the peripheral area of the vehicle,
  • a history image which is an image of a portion obtained by
  • the vehicle periphery image display device and the vehicle periphery image display method according to the present disclosure since the history image is not displayed when the moving speed of the vehicle is larger than the storage speed, an overhead image with a deteriorated image quality is displayed to the driver. It can avoid giving a sense of incongruity.
  • FIG. 1 is a diagram illustrating a vehicle equipped with an overhead image display device.
  • FIG. 2 is an explanatory diagram showing a rough internal configuration of the overhead image display device
  • FIG. 3A is an explanatory diagram showing a state in which a plurality of round marks drawn in a lattice pattern at regular intervals on the ground in front of the vehicle are viewed from above the vehicle
  • FIG. 3B is an explanatory diagram showing a captured image obtained by capturing an area in front of the vehicle with an in-vehicle camera.
  • FIG. 4A is an explanatory diagram of the principle of converting a captured image into an overhead image
  • FIG. 4B is an explanatory diagram of the principle of converting a captured image into an overhead image.
  • FIG. 5 is an explanatory diagram of how to display a bird's-eye view image including the left-right direction of the vehicle using a history image
  • FIG. 6 is an explanatory diagram of a state in which an overhead image is displayed using a history image when the moving speed of the vehicle increases.
  • FIG. 7 is an explanatory diagram illustrating the reason why the image quality of the overhead view image displayed using the history image decreases as the moving speed increases
  • FIG. 8 is a flowchart of the first half of the overhead image display process of the first embodiment.
  • FIG. 9 is a flowchart of the latter half of the overhead image display process of the first embodiment.
  • FIG. 10 is an explanatory diagram illustrating a state in which an overhead image is displayed during deceleration of the vehicle.
  • FIG. 11 is a flowchart of a bird's-eye view image display process according to a modification of the first embodiment.
  • FIG. 12 is an explanatory diagram of a method for acquiring the storage speed according to the deceleration
  • FIG. 13 is an explanatory diagram showing the reason why the storage speed according to the deceleration can be determined.
  • FIG. 14 is a flowchart of the first half of the overhead image display process of the second embodiment.
  • FIG. 15 is a flowchart of the latter half of the overhead image display process of the second embodiment.
  • FIG. 1 shows a vehicle 1 on which an overhead image display device 100 is mounted.
  • the overhead image display device 100 corresponds to the “vehicle peripheral image display device” in the present disclosure.
  • the vehicle 1 includes an in-vehicle camera 10F mounted in front of the vehicle 1, an in-vehicle camera 10R mounted in the rear of the vehicle 1, a display screen 11 mounted in the vehicle interior, and an overhead image display device. 100 and the like.
  • description will be made assuming that the in-vehicle cameras 10F and 10R are mounted in front and rear of the vehicle 1, respectively.
  • the present disclosure can be applied.
  • the bird's-eye view image display device 100 When the bird's-eye view image display device 100 receives an image of the periphery of the vehicle 1 taken by the in-vehicle cameras 10F and 10R, the bird's-eye view image display device 100 converts the image into a bird's-eye view image taken from above the vehicle 1, and then obtains the obtained image. It is displayed on the display screen 11.
  • the vehicle 1 is also equipped with a steering angle sensor 12 for detecting the steering angle of the steering handle 2, a vehicle speed sensor 13 for detecting the vehicle speed, a shift position sensor 14 for detecting a shift position of a transmission (not shown), and the like. Yes.
  • the outputs of the steering angle sensor 12, the vehicle speed sensor 13, and the shift position sensor 14 are input to the overhead view image display device 100, and the overhead view image display device 100 based on these outputs outputs the movement amount (movement speed and The movement information regarding the movement direction) is acquired.
  • the bird's-eye view image display apparatus 100 is based on the movement information of the vehicle 1 and the bird's-eye view images obtained by the on-vehicle cameras 10F and 10R. Is generated on the display screen 11.
  • FIG. 2 shows a rough internal configuration of the overhead image display apparatus 100 of the present embodiment.
  • a bird's-eye view image display apparatus 100 includes a photographed image acquisition unit 101, a photographed bird's-eye view image generation unit 102, a display image generation unit 103, a movement information acquisition unit 104, and a storage speed determination unit 105. And a photographing overhead image storage unit 106.
  • these six “parts” are abstractions obtained by classifying the inside of the overhead view image display device 100 for convenience, focusing on the function of the overhead view image display device 100 displaying an image around the vehicle 1 on the display screen 11. This does not represent that the overhead image display device 100 is physically divided into six parts. Therefore, these “units” can be realized as a computer program executed by the CPU, can be realized as an electronic circuit including an LSI or a memory, and further realized by combining them. You can also.
  • the captured image acquisition unit 101 is connected to the in-vehicle camera 10F and the in-vehicle camera 10R, and acquires a captured image obtained by capturing an area in front of the vehicle 1 from obliquely above from the in-vehicle camera 10F at a constant cycle (about 30 Hz).
  • a captured image obtained by capturing an area behind the vehicle 1 from obliquely above is acquired at a constant period (about 30 Hz).
  • the photographed bird's-eye view image generation unit 102 receives the images taken by the in-vehicle cameras 10F and 10R from the photographed image acquisition unit 101 and generates an overhead image. That is, the photographed image obtained by the in-vehicle camera 10F is an image obtained by photographing an area in front of the vehicle 1 from obliquely above, and the photographed image is obtained from above the vehicle 1 by performing line-of-sight conversion described later on this image. Such a bird's-eye view image is generated. Similarly, the image captured by the in-vehicle camera 10 ⁇ / b> R is an image obtained by capturing an area behind the vehicle 1 from obliquely above, and this image is converted into an overhead view image captured from above the vehicle 1.
  • the shooting overhead image generation unit 102 corresponds to the “shooting overhead image generation unit” in the present disclosure.
  • the bird's-eye view image display device 100 not only the front-rear direction area of the vehicle 1 but also the left-right direction area of the vehicle 1 is generated and displayed on the display screen 11. can do.
  • the bird's-eye view image displayed in the front-rear direction of the vehicle 1 is an image obtained by line-of-sight conversion of the current image captured by the in-vehicle cameras 10F and 10R.
  • the bird's-eye view image displayed in the left-right direction of the vehicle 1 is an image synthesized from the past bird's-eye view images displayed in the front-rear direction of the vehicle 1 based on the movement information of the vehicle 1.
  • the bird's-eye view image displayed in the front-rear direction of the vehicle 1 is referred to as a “shooting bird's-eye view image” in order to distinguish it from the synthesized bird's-eye view image displayed in the left-right direction of the vehicle 1.
  • the term “overhead image” simply refers to an image viewed from above the vehicle 1, and accordingly, both the imaged bird's-eye view image and the history image correspond to the “overhead image”.
  • the photographic overhead image generation unit 102 outputs the generated photographic overhead image to the display image generation unit 103 and the photographic overhead image storage unit 106.
  • the movement information acquisition unit 104 is connected to the steering angle sensor 12, the vehicle speed sensor 13, and the shift position sensor 14.
  • the movement information acquisition unit 104 acquires movement information of the vehicle 1 based on the outputs of these sensors.
  • the movement information of the vehicle 1 is information such as whether the vehicle 1 is moving forward, backward, or stopped, whether the vehicle 1 is moving straight, or whether it is turning right Whether the vehicle is turning left, or if it is turning, information on the magnitude of the turn, information on the moving speed of the vehicle 1, and the like.
  • the movement information acquisition unit 104 outputs the acquired movement information of the vehicle 1 to the storage speed determination unit 105, the photographic overhead image storage unit 106, and the display image generation unit 103.
  • the movement information acquisition unit 104 corresponds to the “movement information acquisition unit” in the present disclosure.
  • the storage speed determination unit 105 determines the “storage speed” based on the movement information of the vehicle 1 received from the movement information acquisition unit 104.
  • the storage speed is a reference speed for determining whether or not the photographic overhead image storage unit 106 stores the photographic overhead image generated by the photographic overhead image generation unit 102.
  • the storage speed determination unit 105 corresponds to the “storage speed determination unit” in the present disclosure.
  • the photographic bird's-eye view image storage unit 106 determines whether or not to store the bird's-eye view image based on the movement information of the vehicle 1 received from the movement information acquisition unit 104 and the storage speed received from the storage speed determination unit 105. For example, when the moving speed of the vehicle 1 is lower than the storage speed received from the storage speed determination unit 105, it is determined that the overhead image is stored. On the other hand, when the moving speed of the vehicle 1 is larger than the storage speed, it is determined that the overhead image is not stored.
  • the photographic overhead image storage unit 106 corresponds to the “photographic overhead image storage unit” in the present disclosure.
  • the display image generation unit 103 When the display image generation unit 103 receives the shooting overhead image from the shooting overhead image generation unit 102, the display image generation unit 103 displays the shooting overhead image on the display screen 11 in a shooting region existing in the front-rear direction of the vehicle 1.
  • the display image generation unit 103 compares the moving speed of the vehicle 1 with a predetermined “display speed”, and if the moving speed is equal to or lower than the display speed, the display image generating unit 103 displays an overhead image of the left and right region of the vehicle 1. It is generated and displayed on a blind spot area existing in the left-right direction of the vehicle 1 on the display screen 11.
  • the overhead view image of the blind spot area is generated by cutting out an image (history image) of a corresponding part based on the movement information of the vehicle 1 from the past shooting overhead image stored in the shooting overhead image storage unit 106.
  • the display image generation unit 103 corresponds to the “shooting overhead image display unit” and the “history image display unit” in the present disclosure.
  • the bird's-eye view image display device 100 displays a bird's-eye view image of the entire circumference including the left-right direction of the vehicle 1 when the moving speed of the vehicle 1 is smaller than a predetermined display speed.
  • a predetermined display speed when the moving speed of the vehicle 1 is higher than the display speed, an overhead image is displayed in the front-rear direction of the vehicle 1, but no overhead image is displayed in the left-right direction.
  • FIG. 3A shows a plurality of round marks drawn in a lattice pattern at equal intervals on the ground in front of the vehicle 1 as viewed from above the vehicle 1.
  • two alternate long and short dash lines extending obliquely from the in-vehicle camera 10F represent the boundary between the imaging region and the blind spot region of the in-vehicle camera 10F.
  • a round mark displayed with a thick solid line indicates that the mark is within the imaging region of the in-vehicle camera 10F
  • a round mark displayed with a thin broken line indicates that the mark is mounted on the vehicle. It represents that it is in the blind spot area of the camera 10F.
  • the in-vehicle camera 10F (or the in-vehicle camera 10R) images such a ground from obliquely above.
  • FIG. 3B shows a captured image obtained by the in-vehicle camera 10F.
  • the photographed image shows a mark on the ground that exists within the photographing range of the in-vehicle camera 10F.
  • the mark a on the photographed image is a photograph of the mark A on the ground in FIG. 3A
  • the mark b on the photographed image is a photograph of the mark B on the ground.
  • the mark c and the mark d on the photographed image are the marks C and D on the ground.
  • the captured image illustrated in FIG. 3B can be converted into an image (overhead image) that looks down on the ground from above the vehicle 1 as illustrated in FIG. 3A. This is the way of thinking when generating a bird's-eye view image by converting the line-of-sight of a captured image of the in-vehicle camera 10F (or in-vehicle camera 10R).
  • the captured image shown in FIG. 4A is converted into an overhead image as shown in FIG. 4B.
  • the bird's-eye view image thus obtained becomes an image that spreads in the left-right direction as the position is further away. This is because the imaging range of the in-vehicle camera 10F (or in-vehicle camera 10R) is widened on both the left and right sides (see FIG. 3A).
  • the region extending to the left in the overhead image is displayed as region L, and the region expanding to the right is displayed as region R.
  • both the region L and the region R are within the imaging region of the in-vehicle camera 10F.
  • the region L becomes a blind spot region in the left direction of the vehicle 1
  • the region R becomes a blind spot region in the right direction of the vehicle 1.
  • a bird's-eye view image (shooting bird's-eye view image) obtained by line-of-sight conversion of the photographed image of the in-vehicle camera 10F is stored, and the current blind spot area is selected from the past photographed bird's-eye images as the vehicle 1 moves.
  • the image (history image) of the part corresponding to is cut out and displayed. In this way, it is possible to display a bird's-eye view image also in the left and right area of the vehicle 1.
  • FIG. 5 illustrates a state in which the vehicle 1 moves with time and a history image is displayed in the left and right areas of the vehicle 1 accordingly.
  • a photographing overhead image is displayed in the photographing regions before and after the vehicle 1.
  • a new shooting overhead image is generated on the basis of the images shot by the in-vehicle cameras 10F and 10R at time Tb, so this new shooting overhead image is displayed.
  • the history image is cut out from the photographed bird's-eye view image displayed in front of the vehicle 1 by the amount of advance of the vehicle 1 and displayed in the left and right blind spot areas of the vehicle 1. Further, the history image that has already been displayed in the blind spot area is moved to the rear of the vehicle 1 by the amount that the vehicle 1 has advanced.
  • the history image is displayed in the entire blind spot area of the vehicle 1, and as a result, the overhead image can be displayed over the entire circumference of the vehicle 1.
  • FIG. 6 illustrates a state in which an overhead image is displayed in the blind spot area of the vehicle 1 using a history image when the moving speed of the vehicle 1 is high.
  • FIG. 6 it is assumed that the history image is not displayed at the time Ta as in FIG.
  • the vehicle 1 in response to the moving speed of the vehicle 1 being large, the vehicle 1 is moving forward more greatly than in the case of FIG. 5, and accordingly, the image displayed in front of the vehicle 1 at time Ta.
  • the size of the history image cut out from the overhead image also increases.
  • the vehicle 1 further moves forward, and a history image (shown as “b” in the figure) is cut out from the photographed bird's-eye view image displayed forward at time Tb. Is displayed.
  • the history image (displayed as “a” in the drawing) displayed in the blind spot area at time Tb moves to the rear of the vehicle 1.
  • a shooting bird's-eye view image displayed in front of the vehicle 1 and a history image (see FIG. Attention is paid to the boundary portion between “b” and “. Similarly, attention is paid to a boundary portion between a new history image (displayed as “b” in the drawing) and an old history image (displayed as “a” in the drawing).
  • FIG. 7 shows the boundary portion of interest in an enlarged manner.
  • the imaged bird's-eye view image of the area adjacent to this boundary is an image obtained by performing line-of-sight conversion on an image obtained by photographing the ground immediately near the vehicle 1.
  • a history image of a region adjacent to the boundary is an image obtained by performing line-of-sight conversion on an image of the ground far away from the vehicle 1.
  • a new history image adjacent to the boundary is an image obtained by line-of-sight conversion of an image near the vehicle 1, whereas an old part adjacent to the boundary is old.
  • a history image is an image obtained by line-of-sight conversion of an image far from the vehicle 1. For this reason, even at this boundary portion, it looks as if two images with greatly different enlargement rates are connected, and the image quality is greatly deteriorated.
  • the history image is not enlarged by shortening the image capturing period of the in-vehicle cameras 10F and 10R as the moving speed of the vehicle 1 increases, such deterioration of the image quality can be avoided. it can.
  • the overhead image display apparatus 100 performs the following overhead image display processing.
  • (Overhead image display processing of the first embodiment) 8 and 9 show a flowchart of the overhead image display process of the first embodiment executed by the overhead image display apparatus 100 of the present embodiment. This process is started when a predetermined operation for displaying an overhead image on the display screen 11 is performed by the driver while the engine of the vehicle 1 is activated.
  • the bird's-eye view image display process (S100) of the first embodiment is started, first, captured images of the front and rear of the vehicle 1 are acquired from the in-vehicle camera 10F and the in-vehicle camera 10R (S101).
  • the captured images acquired from the in-vehicle cameras 10F and 10R are line-of-sight-converted, and the captured overhead images for the front and rear imaging areas of the vehicle 1 are generated (S102).
  • the frame memory is a memory area used for displaying an image on the display screen 11.
  • An image displayed on the display screen 11 is generated on the frame memory.
  • the image is displayed on the display screen 11 by outputting the data in the frame memory as a video signal.
  • Each address on the frame memory corresponds to each pixel on the display screen 11.
  • a front bird's-eye view image is written in each address of the frame memory corresponding to the shooting area of the in-vehicle camera 10F on the display screen 11, and a rear address is written in each address of the frame memory corresponding to the shooting area of the in-vehicle camera 10R.
  • Write a bird's-eye view image is written in each address of the frame memory corresponding to the shooting area of the in-vehicle camera 10F on the display screen 11, and a rear address is written in each address of the frame memory corresponding to the shooting area of the in-vehicle camera 10R.
  • the overhead image display device 100 acquires movement information of the vehicle 1 (S104).
  • the movement information of the vehicle 1 is information acquired from the steering angle sensor 12, the vehicle speed sensor 13, the shift position sensor 14, and the like.
  • the display speed is a reference speed for determining whether to display an overhead image using a history image in the blind spot area of the vehicle 1.
  • the display speed is set to an appropriate speed in advance within a speed range of 10 km to 20 km per hour.
  • this bird's-eye view image display process is started when a driver who wants to display a bird's-eye view image and check the situation around the vehicle 1 performs a predetermined operation.
  • the driver wants to check the situation around the vehicle 1 using the bird's-eye view image mainly when the moving speed of the vehicle 1 is low.
  • this does not necessarily mean that the operation for displaying the overhead view image is performed in a state where the moving speed of the vehicle 1 is low.
  • the vehicle 1 may be displayed in advance while the vehicle 1 is decelerated.
  • the bird's-eye view image display device when the photographed bird's-eye view image is generated and written in the frame memory (S102, S103), it is determined whether or not the moving speed of the vehicle 1 is equal to or lower than the display speed (S105).
  • the display speed of the vehicle 1 is equal to or lower than the display speed (S105: yes)
  • the captured bird's-eye view image is displayed in a state where the shooting timing can be identified (for example, a time stamp).
  • the storage speed is a speed that is a criterion for determining whether or not to store a photographic overhead image in preparation for the generation of a history image.
  • an appropriate speed larger than the display speed is preset as the storage speed.
  • the storage speed is set to a value 5 km / h higher than the display speed.
  • the shooting overhead image is displayed in a state where the shooting timing is identifiable (for example, with a time stamp) in preparation for generation of a history image. Is stored (S107).
  • the moving speed of the vehicle 1 is equal to or lower than the display speed (S108). If the moving speed has not yet decreased below the display speed (S108: no), a history image is displayed in the blind spot area of the vehicle 1. Since it is determined that it is not necessary to display the image, the image on the frame memory is output to the display screen 11 (S114 in FIG. 9). Since the shooting bird's-eye view image is written in the address corresponding to the shooting area on the frame memory in S103, the bird's-eye view image is displayed on the display screen 11 in the shooting areas in front and behind the vehicle 1.
  • Whether or not the vehicle 1 is moving forward can be determined based on the movement information acquired in S104.
  • the obtained history image is written in the frame memory corresponding to the blind spot area of the in-vehicle camera 10F and the in-vehicle camera 10R on the display screen 11 (S113).
  • the overhead image display device 100 outputs the image on the frame memory to the display screen 11 (S114). As a result, a bird's-eye view image as if the periphery of the vehicle 1 was viewed from above is displayed on the display screen 11.
  • FIG. 10 illustrates a state in which an overhead image is displayed on the display screen 11 by the overhead image display processing of the first embodiment described above.
  • an operation for starting the display of an overhead image is performed by the driver while the moving speed of the vehicle 1 is decelerated.
  • the moving speed of the vehicle 1 is higher than the display speed and higher than the storage speed at the time T1 when the operation for starting the display of the overhead image is performed. For this reason, in the above-described bird's-eye view image display processing, it is determined that the moving speed is higher than the display speed (S105: no in FIG. 8) and higher than the storage speed (S106: no). No history image is displayed in the blind spot area. In addition, in the photographing areas before and after the vehicle 1, a photographed bird's-eye view image is displayed in response to an operation for starting display of the bird's-eye view image performed by the driver.
  • the moving speed of the vehicle 1 decreases to the display speed
  • a history image is displayed in the left and right blind spots of the vehicle 1. Therefore, the bird's-eye view image is displayed on the entire circumference of the vehicle 1 together with the shooting bird's-eye view images displayed in the shooting areas before and after the vehicle 1.
  • the vehicle 1 even when the driver performs an operation for displaying the bird's-eye view image at a stage where the moving speed of the vehicle 1 is higher than the display speed, the vehicle 1.
  • the overhead image is not displayed in the blind spot area of the vehicle 1 using the history image until the moving speed becomes equal to or lower than the display speed. For this reason, it is possible to avoid the overhead image having poor image quality as described above with reference to FIGS. 6 and 7 being displayed on the display screen 11.
  • the bird's eye view of the blind spot area of the vehicle 1 is kept until the moving speed of the vehicle 1 becomes lower than the display speed.
  • the image disappears.
  • the driver actually needs to confirm the bird's eye view of the blind spot area when the moving speed of the vehicle 1 is low, such as when passing through a narrow alley or parking in a narrow parking space. Therefore, if the history image is displayed when the moving speed becomes lower than the display speed, there is no practical problem even if the history image is not displayed when the moving speed is higher than the display speed.
  • the photographed overhead image is taken at time T2 (see FIG. 10) when the vehicle 1 is reduced to the storage speed before the moving speed is reduced to the display speed. Memory is started. For this reason, at the time T3 when the moving speed of the vehicle 1 is reduced to the display speed, a photographic overhead view image for cutting out the history image is already stored. As a result, when the moving speed of the vehicle 1 reaches the display speed, a history image of the entire blind spot area of the vehicle 1 can be displayed immediately.
  • the storage speed for storing the captured bird's-eye view image is preset to an appropriate speed.
  • an appropriate storage speed may be set according to the degree of deceleration of the moving speed of the vehicle 1 (hereinafter referred to as deceleration).
  • the moving speed of the vehicle 1 is higher than the display speed (S105: no)
  • the moving speed of the vehicle 1 is It was determined whether or not the speed was lower than the storage speed (S106).
  • Whether or not the vehicle is decelerating can be determined based on movement information of the vehicle 1. As a result, when it is determined that the vehicle 1 is not decelerating (S120: no), the moving speed of the vehicle 1 is far from the display speed, so that the photographed bird's-eye view image is not stored and S114 in FIG. 9 is stored. To output the image on the frame memory toward the display screen 11.
  • the deceleration of the vehicle 1 is calculated (S121).
  • the amount of decrease in the moving speed of the vehicle 1 per unit time is calculated as the deceleration.
  • the storage speed corresponding to the calculated deceleration is acquired (S122).
  • the storage speed for the deceleration can be calculated by the calculation formula shown in FIG.
  • “a” represents deceleration
  • “V1” represents storage speed
  • “VL” represents vehicle length of the vehicle 1.
  • the calculation result of the storage speed with respect to the deceleration may be stored in advance as a table, and the storage speed with respect to the deceleration may be acquired without referring to the table.
  • the process moves to S106 in FIG. 8 to determine whether or not the moving speed of the vehicle 1 is equal to or lower than the storage speed.
  • the storage speed used for the determination at this time is the movement speed previously acquired in S122.
  • the overhead image display is started by the driver's operation at time T1 during deceleration of the vehicle 1. If the vehicle 1 decelerates at the deceleration at that time, the display speed V0 is reached at time T3, so that it is only necessary to start storing the captured overhead image before that.
  • the display speed V0 is determined in advance.
  • the distance that the vehicle 1 moves from time T2 to time T3 is an area hatched in FIG. Therefore, if a time T2 is selected such that this area is equal to the vehicle length VL of the vehicle 1, a photographed overhead image at a distance corresponding to the vehicle length VL is stored at the time T3. A history image of the entire blind spot area of the vehicle 1 can be generated. Further, the speed at time T2 selected in this way becomes the storage speed V1.
  • VL V0 ⁇ T + 1/2 ⁇ a ⁇ T ⁇ 2
  • V1 V0 + a ⁇ T
  • V1 (2 ⁇ VL ⁇ a + V0 ⁇ 2) ⁇ 0.5
  • the storage of the captured overhead image can be started at the storage speed corresponding to the deceleration of the vehicle 1, when the vehicle 1 decreases to the display speed. It is possible to store a photographed bird's-eye view image at a distance corresponding to the vehicle length. As a result, when the display of the history image is started in the blind spot area of the vehicle 1, it is possible to suppress the occurrence of an area in which the history image is not displayed in the blind spot area. It can be easily confirmed.
  • the timing for starting the display of the bird's-eye view image in the blind spot area of the vehicle 1 does not necessarily need to exactly coincide with the timing at which the moving speed of the vehicle 1 is reduced to the display speed. That is, as long as it does not deviate significantly from the display speed, the display of the bird's-eye view image in the blind spot area of the vehicle 1 may be started from a speed higher than the display speed or a speed lower than the display speed.
  • the overhead image display process (S200) of the second embodiment described below is not that the moving speed of the vehicle 1 has decreased to the display speed, but that the vehicle 1 has been stored after the storage of the captured overhead image has started. After confirming that the vehicle has moved by a distance corresponding to the vehicle length, the display of the overhead image using the history image is greatly different.
  • the overhead image display processing (S200) of the second embodiment will be briefly described with reference to the flowchart.
  • the overhead image display processing of the second embodiment is also a predetermined for displaying the overhead image on the display screen 11 in a state where the engine of the vehicle 1 is activated, similarly to the overhead image display processing of the first embodiment described above.
  • the overhead image display device 100 starts.
  • a captured image is acquired from the in-vehicle camera 10F and the in-vehicle camera 10R (S201). Then, the acquired captured image is transformed into a line of sight to generate a captured overhead image (S202), and stored in the memory (with a time stamp, for example) in a state where the imaging timing can be identified (S203).
  • the storage speed is set in advance.
  • the bird's-eye view image display process of the second embodiment it is determined whether or not the distance corresponding to the vehicle length of the vehicle 1 has moved since the storage of the photographed bird's-eye view image is started (S210). As a result, if it is determined that the distance corresponding to the vehicle length has not yet been moved (S210: no), the captured overhead image written in the frame memory is output to the display screen 11 (S215 in FIG. 15).
  • the image in the frame memory is output to the display screen 11 ( S215).
  • the history image is written in this way, and the image on the frame memory is output to the display screen 11 to display the image.
  • the screen 11 displays an overhead image as if the periphery of the vehicle 1 was viewed from above.
  • the display of the overhead image using the history image is started. It is possible to display a bird's-eye view image of the entire one blind spot area.
  • the vehicle periphery image display device and the vehicle periphery image display method according to an example of the present disclosure perform a line-of-sight conversion on a captured image obtained by an in-vehicle camera to generate a captured overhead image. Then, when the movement information including the movement speed of the vehicle is acquired and the movement speed is larger than the predetermined display speed, a captured overhead image is displayed on the display screen at a position corresponding to the peripheral area captured by the in-vehicle camera. indicate. When the moving speed is equal to or lower than the predetermined display speed, the history image is displayed on the display screen at a position corresponding to the blind spot area of the in-vehicle camera.
  • the vehicle periphery image display device and the vehicle periphery image display method of the present disclosure since the history image is not displayed when the moving speed of the vehicle is higher than the display speed, an overhead image with deteriorated image quality is displayed and the driver is displayed. It can avoid giving a sense of incongruity.
  • each step is expressed as S100, for example. Further, each step can be divided into a plurality of sub-steps, while a plurality of steps can be combined into one step.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Mechanical Engineering (AREA)
  • Signal Processing (AREA)
  • Closed-Circuit Television Systems (AREA)
  • Image Processing (AREA)
  • Traffic Control Systems (AREA)
  • Image Analysis (AREA)

Abstract

 There is provided a vehicle peripheral image display device (100) for displaying, on a display screen (11), an image in which the peripheral area of a vehicle (1) is photographed using vehicle-mounted cameras (10F, 10R). The vehicle peripheral image display device (100) is provided with: a photographed overhead view image generation unit (102) for generating a photographed overhead view image; a photographed overhead view image storage unit (106) for storing the photographed overhead view image; a photographed overhead view image display unit (103) for displaying the photographed overhead view image on a display screen (11); a movement information acquisition unit (104) for acquiring movement information; and a history image display unit (103) for displaying a history image. The history image display unit (103) does not display the history image when the movement speed exceeds a prescribed display speed, and displays the history image when the movement speed is lower than or equal to the display speed.

Description

車両周辺画像表示装置、車両周辺画像表示方法Vehicle periphery image display device and vehicle periphery image display method 関連出願の相互参照Cross-reference of related applications
 本出願は、2014年3月5日に出願された日本特許出願2014-42987号に基づいており、その開示は参照により本明細書に組み込まれる。 This application is based on Japanese Patent Application No. 2014-42987 filed on March 5, 2014, the disclosure of which is incorporated herein by reference.
 本開示は、車載カメラを用いて車両の周辺領域を撮影した画像を、表示画面上に表示する技術に関する。 The present disclosure relates to a technique for displaying an image obtained by photographing a peripheral area of a vehicle using an in-vehicle camera on a display screen.
 車両の前後左右に搭載した車載カメラで得られた撮影画像に対して、あたかも車両を上方から見たような画像に視線変換する処理を施して、得られた画像(俯瞰画像)を表示する技術が知られている。車両の運転者は俯瞰画像を確認することができれば、周囲に存在する障害物と自車両との位置関係や、路面に描かれた表示などと自車両との位置関係を、距離感も含めて容易に把握することが可能となる。 Technology that displays the obtained image (overhead image) by performing a process of converting the line of sight of the captured images obtained by the in-vehicle cameras mounted on the front, rear, left and right of the vehicle as if the vehicle was viewed from above It has been known. If the driver of the vehicle can check the bird's-eye view image, the positional relationship between obstacles and the vehicle in the vicinity, the positional relationship between the display drawn on the road surface and the vehicle, including the sense of distance It becomes possible to grasp easily.
 また、車両の左右には車載カメラを搭載しないにも拘わらず、車両の前後左右方向の周辺領域の俯瞰画像を表示する技術も提案されている(特許文献1)。この技術では、車両の左右方向の周辺領域については車載カメラで直接画像を撮影することはできないが、車両の前後に搭載した車載カメラの画像から、次のような原理で左右方向の俯瞰画像を生成する。 In addition, a technique for displaying an overhead image of a peripheral region in the front-rear and left-right directions of the vehicle has been proposed (Patent Document 1) even though the vehicle-mounted cameras are not mounted on the left and right sides of the vehicle. With this technology, it is not possible to directly capture images in the left and right peripheral areas of the vehicle with the in-vehicle camera, but from the images of the in-vehicle cameras mounted on the front and rear of the vehicle, the overhead view image in the left and right direction is obtained according to the following principle. Generate.
 先ず、車両の前方の車載カメラによって得られた撮影画像には、車両の正面の領域だけでなく、その両側に斜め前方の領域も写っている。従って、前方の車載カメラから得られる俯瞰画像にも、車両の左右斜め前方の領域の画像が含まれている。そして、この斜め前方の領域は、車両が前進することによって、やがては車両の左右方向の領域となる。このことから、俯瞰画像を記憶しておき、車両の移動量に応じて、記憶しておいた俯瞰画像の中から車両の左右方向の領域に対応する画像を切り出して表示すれば、車両の左右方向も含めて周辺領域の俯瞰画像を表示することができる。尚、車両の左右方向に表示される俯瞰画像は、実際には少し前に撮影されていた過去の俯瞰画像である。このため、過去の俯瞰画像から切り出されて車両の左右方向に表示される画像は「履歴画像」と呼ばれる。 First, in the captured image obtained by the in-vehicle camera in front of the vehicle, not only the area in front of the vehicle but also areas on the diagonally forward are shown on both sides. Therefore, the bird's-eye view image obtained from the front vehicle-mounted camera also includes an image of a region diagonally forward in the left and right directions of the vehicle. This diagonally forward area eventually becomes an area in the left-right direction of the vehicle as the vehicle moves forward. From this, if the bird's-eye view image is stored and an image corresponding to the region in the left-right direction of the vehicle is cut out and displayed from the stored bird's-eye view image according to the amount of movement of the vehicle, A bird's-eye view of the surrounding area including the direction can be displayed. Note that the bird's-eye view image displayed in the left-right direction of the vehicle is a past bird's-eye view image that was actually taken a little while ago. For this reason, an image cut out from a past bird's-eye view image and displayed in the left-right direction of the vehicle is called a “history image”.
 また、こうして画面上に表示された俯瞰画像は、車載カメラが新たな画像を撮影する度に更新されており、車載カメラが新たな画像を撮影する周期は一定である。従って、車両の移動速度が分かれば、前回に撮影した画像中でどれだけの領域が死角領域に移動するかを知ることができる。更には、前回に撮影した画像だけでは全ての死角領域に履歴画像を表示することはできないので、死角領域中で前々回に撮影した画像あるいはそれより前に撮影した画像を履歴画像として表示すればよい。 In addition, the overhead image displayed on the screen in this way is updated every time the in-vehicle camera captures a new image, and the cycle in which the in-vehicle camera captures a new image is constant. Therefore, if the moving speed of the vehicle is known, it is possible to know how many areas move to the blind spot area in the previously captured image. Furthermore, since it is not possible to display a history image in all blind spot areas only with the last photographed image, it is only necessary to display an image photographed two times before or in the blind spot area as a history image. .
 もちろん、上述した内容は、車両が後進している場合にも全く同様に当て嵌まる。従って、車両の後進時にも、車両の前後左右方向の俯瞰画像を表示することができる。 Of course, the above description applies in exactly the same manner when the vehicle is moving backward. Accordingly, it is possible to display a bird's-eye view image in the front-rear and left-right directions of the vehicle even when the vehicle is moving backward.
 本願発明者らは、俯瞰画像を表示する技術に関して以下を見出した。 The inventors of the present application have found the following regarding a technique for displaying an overhead image.
 しかし、履歴画像を利用して車載カメラの死角となる領域の俯瞰画像を表示する技術は、車両の移動速度が高くなると表示画質が低下して、運転者に違和感を与える虞がある。これは、移動速度が高くなると、車載カメラで撮影した画像中で遠くが写った部分を視線変換して履歴画像として表示しなければならなくなり、遠くを写したことによる画像の分解能の不足や、視線変換による画像の歪みの影響が目立ち易くなるためである。 However, the technology for displaying a bird's-eye view image of an area that becomes a blind spot of the in-vehicle camera using the history image may cause the display image quality to deteriorate as the moving speed of the vehicle increases, causing the driver to feel uncomfortable. This means that if the moving speed increases, the part of the image taken with the in-vehicle camera must be line-of-sight converted and displayed as a history image. This is because the influence of image distortion due to line-of-sight conversion is easily noticeable.
 もちろん、車載カメラが新たな画像を撮影する周期(従って、画面上で俯瞰画像を更新する周期)を短くすればこうした弊害は回避することができるが、俯瞰画像の更新周期を短くすると処理負担が増加する虞がある。 Of course, this problem can be avoided by shortening the period when the in-vehicle camera captures a new image (and thus the period when the overhead image is updated on the screen), but if the update period of the overhead image is shortened, the processing burden is reduced. May increase.
日本国公開特許公報2002-373327号Japanese Published Patent Publication No. 2002-373327
 本開示は、車両の移動速度が高い場合でも運転者に違和感を与えることなく、車両の周辺の俯瞰画像を表示することが可能な技術の提供を目的とする。 This disclosure is intended to provide a technique capable of displaying a bird's-eye view of the surroundings of a vehicle without causing the driver to feel uncomfortable even when the moving speed of the vehicle is high.
 本開示の一例に係る車両周辺画像表示装置は、車載カメラを用いて車両の周辺領域を撮影した画像を表示画面上に表示する車両周辺画像表示装置であって、前記車載カメラによって得られた撮影画像に対して、該車載カメラに写った前記周辺領域を前記車両の上方から撮影した画像に変換する視線変換を施すことによって、撮影俯瞰画像を生成する撮影俯瞰画像生成部と、前記撮影俯瞰画像を記憶する撮影俯瞰画像記憶部と、前記表示画面上で、前記車載カメラが撮影した前記周辺領域に対応する位置に前記撮影俯瞰画像を表示する撮影俯瞰画像表示部と、前記車両の移動速度を含む移動情報を取得する移動情報取得部と、前記車両の周辺領域の中で前記車載カメラの死角となる死角領域については、前記車両が現在の位置に達する前に生成された前記撮影俯瞰画像の中から該死角領域を撮影した部分の画像である履歴画像を、前記移動情報に基づいて切り出して、前記表示画面上の該死角領域に対応する位置に表示する履歴画像表示部とを備え、前記撮影俯瞰画像記憶部は、前記移動速度が所定の記憶速度以下になると前記撮影俯瞰画像の記憶を開始する部であり、前記履歴画像表示部は、前記撮影俯瞰画像の記憶を開始してから前記車両が所定距離を移動した後に、前記履歴画像を表示する。 A vehicle peripheral image display device according to an example of the present disclosure is a vehicle peripheral image display device that displays an image of a peripheral region of a vehicle using a vehicle-mounted camera on a display screen, and the imaging obtained by the vehicle-mounted camera. A photographic bird's-eye view image generation unit that generates a photographic bird's-eye view image by performing line-of-sight conversion for converting the peripheral region captured by the in-vehicle camera into an image photographed from above the vehicle. A shooting bird's-eye view image storage unit for storing the shooting bird's-eye view image display unit for displaying the shooting bird's-eye view image at a position corresponding to the peripheral area shot by the vehicle-mounted camera on the display screen, and a moving speed of the vehicle. A movement information acquisition unit that acquires movement information including a blind spot area that is a blind spot of the in-vehicle camera in the peripheral area of the vehicle is generated before the vehicle reaches the current position. A history image that is an image of a portion of the photographed bird's-eye image captured from the blind spot area is cut out based on the movement information and displayed at a position corresponding to the blind spot area on the display screen. A photographing unit, and the recording bird's-eye image storage unit is a unit that starts storing the photographing bird's-eye image when the moving speed is equal to or lower than a predetermined storage speed. The history image is displayed after the vehicle has moved a predetermined distance from the start of storage.
 また、本開示の別の一例に係る車両周辺画像表示方法は、車載カメラを用いて車両の周辺領域を撮影した画像を表示画面上に表示する車両周辺画像表示装置であって、前記車載カメラによって得られた撮影画像に対して、該車載カメラに写った前記周辺領域を前記車両の上方から撮影した画像に変換する視線変換を施すことによって、撮影俯瞰画像を生成する撮影俯瞰画像生成工程と、前記撮影俯瞰画像を記憶する撮影俯瞰画像記憶工程と、前記表示画面上で、前記車載カメラが撮影した前記周辺領域に対応する位置に前記撮影俯瞰画像を表示する撮影俯瞰画像表示工程と、前記車両の移動速度を含む移動情報を取得する移動情報取得工程と、前記車両の周辺領域の中で前記車載カメラの死角となる死角領域については、前記車両が現在の位置に達する前に生成された前記撮影俯瞰画像の中から該死角領域を撮影した部分の画像である履歴画像を、前記移動情報に基づいて切り出して、前記表示画面上の該死角領域に対応する位置に表示する履歴画像表示工程とを備え、前記撮影俯瞰画像記憶工程は、前記移動速度が所定の記憶速度以下になると前記撮影俯瞰画像の記憶を開始する工程であり、前記履歴画像表示工程は、前記撮影俯瞰画像の記憶を開始してから前記車両が所定距離を移動した後に、前記履歴画像を表示する工程である。 In addition, a vehicle peripheral image display method according to another example of the present disclosure is a vehicle peripheral image display device that displays an image obtained by photographing a peripheral region of a vehicle using a vehicle-mounted camera on a display screen. A shooting bird's-eye view image generating step for generating a shooting bird's-eye view image by performing line-of-sight conversion for converting the peripheral area captured by the vehicle-mounted camera into an image shot from above the vehicle with respect to the obtained shot image; A shooting bird's-eye view image storing step for storing the shooting bird's-eye view image, a shooting bird's-eye view image displaying step for displaying the shooting bird's-eye view image at a position corresponding to the peripheral area shot by the vehicle-mounted camera on the display screen, and the vehicle A movement information acquisition step of acquiring movement information including a movement speed of the vehicle, and a blind spot area that is a blind spot of the in-vehicle camera in the peripheral area of the vehicle, A history image, which is an image of a portion obtained by photographing the blind spot area, is extracted from the shooting overhead image generated before reaching the position based on the movement information, and corresponds to the blind spot area on the display screen. A history image display step for displaying at a position, wherein the shooting bird's-eye view image storing step is a step of starting storage of the shooting bird's-eye view image when the moving speed falls below a predetermined storage speed, The step of displaying the history image after the vehicle has moved a predetermined distance after the storage of the photographed bird's-eye view image is started.
 本開示の車両周辺画像表示装置および車両周辺画像表示方法においても、車両の移動速度が記憶速度よりも大きい場合には履歴画像が表示されないので、画質の悪化した俯瞰画像が表示されて運転者に違和感を与えることを回避できる。 Also in the vehicle periphery image display device and the vehicle periphery image display method according to the present disclosure, since the history image is not displayed when the moving speed of the vehicle is larger than the storage speed, an overhead image with a deteriorated image quality is displayed to the driver. It can avoid giving a sense of incongruity.
 本開示についての上記目的および他の目的、特徴や利点は、添付の図面を参照した下記の詳細な説明から、より明確になる。添付図面において
図1は、俯瞰画像表示装置を搭載した車両を示す図であり、 図2は、俯瞰画像表示装置の大まかな内部構成を示す説明図であり、 図3Aは、車両の前方の地面に等間隔で格子状に描かれた複数の丸い目印を、車両の上方から見た状態を示した説明図であり、 図3Bは、車載カメラで車両の前方の領域を撮影して得られる撮影画像を示した説明図であり、 図4Aは、撮影画像を俯瞰画像に変換する原理についての説明図であり、 図4Bは、撮影画像を俯瞰画像に変換する原理についての説明図であり、 図5は、履歴画像を用いて車両の左右方向を含めた俯瞰画像を表示する様子についての説明図であり、 図6は、車両の移動速度が大きくなった場合に、履歴画像を用いて俯瞰画像を表示する様子についての説明図であり、 図7は、移動速度が大きくなると履歴画像を用いて表示する俯瞰画像の画質が低下する理由についての説明図であり、 図8は、第1実施例の俯瞰画像表示処理の前半部分のフローチャートであり、 図9は、第1実施例の俯瞰画像表示処理の後半部分のフローチャートであり、 図10は、車両の減速中に俯瞰画像を表示する様子を例示した説明図であり、 図11は、第1実施例の変形例の俯瞰画像表示処理のフローチャートであり、 図12は、減速度に応じて記憶速度を取得する方法についての説明図であり、 図13は、減速度に応じた記憶速度を決定可能な理由についての説明図であり、 図14は、第2実施例の俯瞰画像表示処理の前半部分のフローチャートであり、 図15は、第2実施例の俯瞰画像表示処理の後半部分のフローチャートである。
The above and other objects, features and advantages of the present disclosure will become more apparent from the following detailed description with reference to the accompanying drawings. In the attached drawings
FIG. 1 is a diagram illustrating a vehicle equipped with an overhead image display device. FIG. 2 is an explanatory diagram showing a rough internal configuration of the overhead image display device, FIG. 3A is an explanatory diagram showing a state in which a plurality of round marks drawn in a lattice pattern at regular intervals on the ground in front of the vehicle are viewed from above the vehicle, FIG. 3B is an explanatory diagram showing a captured image obtained by capturing an area in front of the vehicle with an in-vehicle camera. FIG. 4A is an explanatory diagram of the principle of converting a captured image into an overhead image, FIG. 4B is an explanatory diagram of the principle of converting a captured image into an overhead image. FIG. 5 is an explanatory diagram of how to display a bird's-eye view image including the left-right direction of the vehicle using a history image, FIG. 6 is an explanatory diagram of a state in which an overhead image is displayed using a history image when the moving speed of the vehicle increases. FIG. 7 is an explanatory diagram illustrating the reason why the image quality of the overhead view image displayed using the history image decreases as the moving speed increases, FIG. 8 is a flowchart of the first half of the overhead image display process of the first embodiment. FIG. 9 is a flowchart of the latter half of the overhead image display process of the first embodiment. FIG. 10 is an explanatory diagram illustrating a state in which an overhead image is displayed during deceleration of the vehicle. FIG. 11 is a flowchart of a bird's-eye view image display process according to a modification of the first embodiment. FIG. 12 is an explanatory diagram of a method for acquiring the storage speed according to the deceleration, FIG. 13 is an explanatory diagram showing the reason why the storage speed according to the deceleration can be determined. FIG. 14 is a flowchart of the first half of the overhead image display process of the second embodiment. FIG. 15 is a flowchart of the latter half of the overhead image display process of the second embodiment.
 以下では、上述した本開示の内容を明確にするために実施例について説明する。
(装置構成)
 図1には、俯瞰画像表示装置100を搭載した車両1が示されている。尚、本実施例では、俯瞰画像表示装置100が、本開示における「車両周辺画像表示装置」に対応する。
Hereinafter, examples will be described in order to clarify the contents of the present disclosure described above.
(Device configuration)
FIG. 1 shows a vehicle 1 on which an overhead image display device 100 is mounted. In this embodiment, the overhead image display device 100 corresponds to the “vehicle peripheral image display device” in the present disclosure.
 図示されるように車両1は、車両1の前方に搭載された車載カメラ10Fと、車両1の後方に搭載された車載カメラ10Rと、車室内に搭載された表示画面11と、俯瞰画像表示装置100などを備えている。尚、本実施例では、車両1の前方および後方にそれぞれ車載カメラ10F,10Rが搭載されているものとして説明するが、車載カメラ10Fあるいは車載カメラ10Rの何れか一方を搭載した車両1に対しても、本開示を適用することができる。 As illustrated, the vehicle 1 includes an in-vehicle camera 10F mounted in front of the vehicle 1, an in-vehicle camera 10R mounted in the rear of the vehicle 1, a display screen 11 mounted in the vehicle interior, and an overhead image display device. 100 and the like. In the present embodiment, description will be made assuming that the in- vehicle cameras 10F and 10R are mounted in front and rear of the vehicle 1, respectively. However, for the vehicle 1 in which either the in-vehicle camera 10F or the in-vehicle camera 10R is mounted. Also, the present disclosure can be applied.
 俯瞰画像表示装置100は、車載カメラ10F,10Rで撮影した車両1の周辺の画像を受け取ると、その画像を、車両1の上方から撮影したような俯瞰画像に変換した後、得られた画像を表示画面11に表示する。 When the bird's-eye view image display device 100 receives an image of the periphery of the vehicle 1 taken by the in- vehicle cameras 10F and 10R, the bird's-eye view image display device 100 converts the image into a bird's-eye view image taken from above the vehicle 1, and then obtains the obtained image. It is displayed on the display screen 11.
 また、車両1には、操舵ハンドル2の操舵角を検出する操舵角センサー12や、車速を検出する車速センサー13や、図示しない変速機のシフトポジションを検出するシフト位置センサー14などが搭載されている。操舵角センサー12や、車速センサー13、シフト位置センサー14の出力は俯瞰画像表示装置100に入力されており、俯瞰画像表示装置100は、これらの出力に基づいて車両1の移動量(移動速度および移動方向)に関する移動情報を取得する。そして、俯瞰画像表示装置100は、車両1の移動情報と、車載カメラ10F,10Rによって得られた俯瞰画像とに基づいて、車載カメラ10F,10Rの死角になっている領域(車両1の左右方向の領域)についての俯瞰画像を生成して、表示画面11に表示する。 The vehicle 1 is also equipped with a steering angle sensor 12 for detecting the steering angle of the steering handle 2, a vehicle speed sensor 13 for detecting the vehicle speed, a shift position sensor 14 for detecting a shift position of a transmission (not shown), and the like. Yes. The outputs of the steering angle sensor 12, the vehicle speed sensor 13, and the shift position sensor 14 are input to the overhead view image display device 100, and the overhead view image display device 100 based on these outputs outputs the movement amount (movement speed and The movement information regarding the movement direction) is acquired. And the bird's-eye view image display apparatus 100 is based on the movement information of the vehicle 1 and the bird's-eye view images obtained by the on- vehicle cameras 10F and 10R. Is generated on the display screen 11.
 図2には、本実施例の俯瞰画像表示装置100の大まかな内部構成が示されている。図示されるように本実施例の俯瞰画像表示装置100は、撮影画像取得部101と、撮影俯瞰画像生成部102と、表示画像生成部103、移動情報取得部104と、記憶速度決定部105と、撮影俯瞰画像記憶部106とを備えている。 FIG. 2 shows a rough internal configuration of the overhead image display apparatus 100 of the present embodiment. As shown in the figure, a bird's-eye view image display apparatus 100 according to the present embodiment includes a photographed image acquisition unit 101, a photographed bird's-eye view image generation unit 102, a display image generation unit 103, a movement information acquisition unit 104, and a storage speed determination unit 105. And a photographing overhead image storage unit 106.
 尚、これら6つの「部」は、俯瞰画像表示装置100が車両1の周辺の画像を表示画面11に表示する機能に着目して、俯瞰画像表示装置100の内部を便宜的に分類した抽象的な概念であり、俯瞰画像表示装置100が物理的に6つの部分に区分されることを表すものではない。従って、これらの「部」は、CPUで実行されるコンピュータープログラムとして実現することもできるし、LSIやメモリーを含む電子回路として実現することもできるし、更にはこれらを組合せることによって実現することもできる。 Note that these six “parts” are abstractions obtained by classifying the inside of the overhead view image display device 100 for convenience, focusing on the function of the overhead view image display device 100 displaying an image around the vehicle 1 on the display screen 11. This does not represent that the overhead image display device 100 is physically divided into six parts. Therefore, these “units” can be realized as a computer program executed by the CPU, can be realized as an electronic circuit including an LSI or a memory, and further realized by combining them. You can also.
 撮影画像取得部101は、車載カメラ10Fおよび車載カメラ10Rに接続されており、車載カメラ10Fから車両1の前方の領域を斜め上方から撮影した撮影画像を一定周期(約30Hz)で取得する。また、車載カメラ10Rからは、車両1の後方の領域を斜め上方から撮影した撮影画像を一定周期(約30Hz)で取得する。 The captured image acquisition unit 101 is connected to the in-vehicle camera 10F and the in-vehicle camera 10R, and acquires a captured image obtained by capturing an area in front of the vehicle 1 from obliquely above from the in-vehicle camera 10F at a constant cycle (about 30 Hz). In addition, from the in-vehicle camera 10 </ b> R, a captured image obtained by capturing an area behind the vehicle 1 from obliquely above is acquired at a constant period (about 30 Hz).
 撮影俯瞰画像生成部102は、撮影画像取得部101から車載カメラ10F,10Rによる撮影画像を受け取って俯瞰画像を生成する。すなわち、車載カメラ10Fによって得られる撮影画像は、車両1の前方の領域を斜め上方から撮影した画像であるが、この画像に対して後述する視線変換を施すことにより、車両1の上方から撮影したような俯瞰画像を生成する。同様に、車載カメラ10Rによる撮影画像は、車両1の後方の領域を斜め上方から撮影した画像であるが、この画像を、車両1の上方から撮影したような俯瞰画像に変換する。車載カメラ10F,10Rによる撮影画像に視線変換を施して俯瞰画像を生成する方法については後述する。本実施例では、撮影俯瞰画像生成部102が本開示における「撮影俯瞰画像生成部」に対応する。 The photographed bird's-eye view image generation unit 102 receives the images taken by the in- vehicle cameras 10F and 10R from the photographed image acquisition unit 101 and generates an overhead image. That is, the photographed image obtained by the in-vehicle camera 10F is an image obtained by photographing an area in front of the vehicle 1 from obliquely above, and the photographed image is obtained from above the vehicle 1 by performing line-of-sight conversion described later on this image. Such a bird's-eye view image is generated. Similarly, the image captured by the in-vehicle camera 10 </ b> R is an image obtained by capturing an area behind the vehicle 1 from obliquely above, and this image is converted into an overhead view image captured from above the vehicle 1. A method for generating a bird's-eye view image by performing line-of-sight conversion on images captured by the in- vehicle cameras 10F and 10R will be described later. In the present embodiment, the shooting overhead image generation unit 102 corresponds to the “shooting overhead image generation unit” in the present disclosure.
 尚、後述するように、本実施例の俯瞰画像表示装置100では、車両1の前後方向の領域だけでなく、車両1の左右方向の領域についても俯瞰画像を生成して、表示画面11に表示することができる。ここで、車両1の前後方向に表示する俯瞰画像は、車載カメラ10F,10Rで撮影した現在の画像を視線変換した画像である。これに対し、車両1の左右方向に表示する俯瞰画像は、車両1の移動情報に基づいて、車両1の前後方向に表示した過去の俯瞰画像から合成した画像である。そこで以下では、車両1の前後方向に表示する俯瞰画像を、車両1の左右方向に表示する合成された俯瞰画像と区別するために「撮影俯瞰画像」と称することにする。また、単に「俯瞰画像」という時は、車両1の上方から見た画像を意味しており、従って、撮影俯瞰画像も履歴画像も「俯瞰画像」に該当する。 As will be described later, in the bird's-eye view image display device 100 according to the present embodiment, not only the front-rear direction area of the vehicle 1 but also the left-right direction area of the vehicle 1 is generated and displayed on the display screen 11. can do. Here, the bird's-eye view image displayed in the front-rear direction of the vehicle 1 is an image obtained by line-of-sight conversion of the current image captured by the in- vehicle cameras 10F and 10R. On the other hand, the bird's-eye view image displayed in the left-right direction of the vehicle 1 is an image synthesized from the past bird's-eye view images displayed in the front-rear direction of the vehicle 1 based on the movement information of the vehicle 1. Therefore, hereinafter, the bird's-eye view image displayed in the front-rear direction of the vehicle 1 is referred to as a “shooting bird's-eye view image” in order to distinguish it from the synthesized bird's-eye view image displayed in the left-right direction of the vehicle 1. The term “overhead image” simply refers to an image viewed from above the vehicle 1, and accordingly, both the imaged bird's-eye view image and the history image correspond to the “overhead image”.
 撮影俯瞰画像生成部102は、生成した撮影俯瞰画像を、表示画像生成部103と、撮影俯瞰画像記憶部106とに出力する。 The photographic overhead image generation unit 102 outputs the generated photographic overhead image to the display image generation unit 103 and the photographic overhead image storage unit 106.
 移動情報取得部104は、操舵角センサー12や、車速センサー13や、シフト位置センサー14に接続されている。移動情報取得部104は、これらセンサーの出力に基づいて、車両1の移動情報を取得する。ここで、車両1の移動情報とは、車両1が前進しているのか、後進しているのか、停車しているのかといった情報や、車両1が真っ直ぐ進んでいるのか、右旋回しているのか、左旋回しているのか、旋回している場合には旋回の大きさに関する情報や、車両1の移動速度に関する情報などである。 The movement information acquisition unit 104 is connected to the steering angle sensor 12, the vehicle speed sensor 13, and the shift position sensor 14. The movement information acquisition unit 104 acquires movement information of the vehicle 1 based on the outputs of these sensors. Here, the movement information of the vehicle 1 is information such as whether the vehicle 1 is moving forward, backward, or stopped, whether the vehicle 1 is moving straight, or whether it is turning right Whether the vehicle is turning left, or if it is turning, information on the magnitude of the turn, information on the moving speed of the vehicle 1, and the like.
 移動情報取得部104は、取得した車両1の移動情報を、記憶速度決定部105と、撮影俯瞰画像記憶部106と、表示画像生成部103とに出力する。本実施例では、移動情報取得部104が、本開示における「移動情報取得部」に対応する。 The movement information acquisition unit 104 outputs the acquired movement information of the vehicle 1 to the storage speed determination unit 105, the photographic overhead image storage unit 106, and the display image generation unit 103. In the present embodiment, the movement information acquisition unit 104 corresponds to the “movement information acquisition unit” in the present disclosure.
 記憶速度決定部105は、移動情報取得部104から受け取った車両1の移動情報に基づいて、「記憶速度」を決定する。ここで記憶速度とは、撮影俯瞰画像生成部102で生成した撮影俯瞰画像を、撮影俯瞰画像記憶部106で記憶するか否かを判断するための基準となる速度である。 The storage speed determination unit 105 determines the “storage speed” based on the movement information of the vehicle 1 received from the movement information acquisition unit 104. Here, the storage speed is a reference speed for determining whether or not the photographic overhead image storage unit 106 stores the photographic overhead image generated by the photographic overhead image generation unit 102.
 車両1の移動速度が記憶速度よりも大きい場合には、撮影俯瞰画像は記憶せず、移動速度が記憶速度よりも小さくなると撮影俯瞰画像を記憶する。車両1の移動情報から記憶速度を決定する方法については後述する。尚、本実施例では、記憶速度決定部105が、本開示における「記憶速度決定部」に対応する。 When the moving speed of the vehicle 1 is higher than the storage speed, the shooting overhead image is not stored, and when the moving speed is lower than the storage speed, the shooting overhead image is stored. A method for determining the storage speed from the movement information of the vehicle 1 will be described later. In this embodiment, the storage speed determination unit 105 corresponds to the “storage speed determination unit” in the present disclosure.
 撮影俯瞰画像記憶部106は、移動情報取得部104から受け取った車両1の移動情報と、記憶速度決定部105から受け取った記憶速度とに基づいて、俯瞰画像を記憶するか否かを判断する。例えば、車両1の移動速度が、記憶速度決定部105から受け取った記憶速度よりも小さかった場合には、俯瞰画像を記憶すると判断する。これに対して、車両1の移動速度が記憶速度よりも大きかった場合には、俯瞰画像を記憶しないと判断する。 The photographic bird's-eye view image storage unit 106 determines whether or not to store the bird's-eye view image based on the movement information of the vehicle 1 received from the movement information acquisition unit 104 and the storage speed received from the storage speed determination unit 105. For example, when the moving speed of the vehicle 1 is lower than the storage speed received from the storage speed determination unit 105, it is determined that the overhead image is stored. On the other hand, when the moving speed of the vehicle 1 is larger than the storage speed, it is determined that the overhead image is not stored.
 そして、俯瞰画像を記憶すると判断した場合に、撮影俯瞰画像生成部102から受け取った撮影俯瞰画像を記憶する。尚、本実施例では、撮影俯瞰画像記憶部106が、本開示における「撮影俯瞰画像記憶部」に対応する。 Then, when it is determined to store the bird's-eye view image, the photographed bird's-eye view image received from the shooting bird's-eye view image generation unit 102 is stored. In the present embodiment, the photographic overhead image storage unit 106 corresponds to the “photographic overhead image storage unit” in the present disclosure.
 表示画像生成部103は、撮影俯瞰画像生成部102からの撮影俯瞰画像を受け取ると、その撮影俯瞰画像を、表示画面11上で車両1の前後方向に存在する撮影領域に表示する。 When the display image generation unit 103 receives the shooting overhead image from the shooting overhead image generation unit 102, the display image generation unit 103 displays the shooting overhead image on the display screen 11 in a shooting region existing in the front-rear direction of the vehicle 1.
 また、表示画像生成部103は、車両1の移動速度を所定の「表示速度」と比較して、移動速度が表示速度以下であった場合に、車両1の左右方向の領域についての俯瞰画像を生成して、表示画面11上で車両1の左右方向に存在する死角領域に表示する。死角領域の俯瞰画像は、撮影俯瞰画像記憶部106に記憶されている過去の撮影俯瞰画像の中から、車両1の移動情報に基づいて該当する部分の画像(履歴画像)を切り出すことによって生成する。本実施例では、表示画像生成部103が、本開示における「撮影俯瞰画像表示部」および「履歴画像表示部」に対応する。 Further, the display image generation unit 103 compares the moving speed of the vehicle 1 with a predetermined “display speed”, and if the moving speed is equal to or lower than the display speed, the display image generating unit 103 displays an overhead image of the left and right region of the vehicle 1. It is generated and displayed on a blind spot area existing in the left-right direction of the vehicle 1 on the display screen 11. The overhead view image of the blind spot area is generated by cutting out an image (history image) of a corresponding part based on the movement information of the vehicle 1 from the past shooting overhead image stored in the shooting overhead image storage unit 106. . In the present embodiment, the display image generation unit 103 corresponds to the “shooting overhead image display unit” and the “history image display unit” in the present disclosure.
 以上に説明したように、本実施例の俯瞰画像表示装置100は、車両1の移動速度が所定の表示速度より小さい場合には、車両1の左右方向を含めた全周の俯瞰画像を表示する。しかし、車両1の移動速度が表示速度よりも大きい場合には、車両1の前後方向については俯瞰画像を表示するが、左右方向については俯瞰画像を表示しないこととしている。こうすることにより、車両の移動速度が大きい時に俯瞰画像を表示して、運転者に違和感を与えることを回避することが可能となる。 As described above, the bird's-eye view image display device 100 according to the present embodiment displays a bird's-eye view image of the entire circumference including the left-right direction of the vehicle 1 when the moving speed of the vehicle 1 is smaller than a predetermined display speed. . However, when the moving speed of the vehicle 1 is higher than the display speed, an overhead image is displayed in the front-rear direction of the vehicle 1, but no overhead image is displayed in the left-right direction. By doing so, it is possible to avoid giving an uncomfortable feeling to the driver by displaying an overhead image when the moving speed of the vehicle is high.
 以下では、理由、および本実施例の俯瞰画像表示装置100が俯瞰画像を表示する処理について説明する。車載カメラ10F,10Rによる撮影画像を視線変換して撮影俯瞰画像を生成する方法について説明する。
(履歴画像を用いて車両の周辺の俯瞰画像を表示する原理)
 車両1の車載カメラ10F(あるいは車載カメラ10R)で撮影した地面に、複数の目印が等間隔で格子状に描かれていた場合について考える。
Below, the reason and the process which the bird's-eye view image display apparatus 100 of a present Example displays a bird's-eye view image are demonstrated. A method for generating a photographic overhead view image by converting the lines of sight of the images taken by the in- vehicle cameras 10F and 10R will be described.
(Principle of displaying a bird's-eye view around the vehicle using history images)
Consider a case where a plurality of landmarks are drawn in a grid pattern at equal intervals on the ground imaged by the in-vehicle camera 10F (or in-vehicle camera 10R) of the vehicle 1.
 図3Aには、車両1の前方の地面に等間隔で格子状に描かれた複数の丸い目印を、車両1の上方から見た状態で示されている。図中で、車載カメラ10Fから斜め方向に伸びる2本の一点鎖線は、車載カメラ10Fの撮影領域と死角領域との境界を表している。また、丸い目印が太い実線で表示されているのは、その目印が車載カメラ10Fの撮影領域内にあることを表しており、丸い目印が細い破線で表示されているのは、その目印が車載カメラ10Fの死角領域内にあることを表している。車載カメラ10F(あるいは車載カメラ10R)は、このような地面を斜め上方から撮影する。 FIG. 3A shows a plurality of round marks drawn in a lattice pattern at equal intervals on the ground in front of the vehicle 1 as viewed from above the vehicle 1. In the figure, two alternate long and short dash lines extending obliquely from the in-vehicle camera 10F represent the boundary between the imaging region and the blind spot region of the in-vehicle camera 10F. In addition, a round mark displayed with a thick solid line indicates that the mark is within the imaging region of the in-vehicle camera 10F, and a round mark displayed with a thin broken line indicates that the mark is mounted on the vehicle. It represents that it is in the blind spot area of the camera 10F. The in-vehicle camera 10F (or the in-vehicle camera 10R) images such a ground from obliquely above.
 図3Bには、車載カメラ10Fで得られる撮影画像が示されている。撮影画像には、車載カメラ10Fの撮影範囲内に存在する地面の目印が写っている。例えば、撮影画像上での目印aは、図3Aの地面上の目印Aが写ったものであり、撮影画像上での目印bは、地面上の目印Bが写ったものである。同様に、撮影画像上での目印cおよび目印dは、地面上の目印Cおよび目印Dが写ったものである。 FIG. 3B shows a captured image obtained by the in-vehicle camera 10F. The photographed image shows a mark on the ground that exists within the photographing range of the in-vehicle camera 10F. For example, the mark a on the photographed image is a photograph of the mark A on the ground in FIG. 3A, and the mark b on the photographed image is a photograph of the mark B on the ground. Similarly, the mark c and the mark d on the photographed image are the marks C and D on the ground.
 このように、車載カメラ10Fの撮影範囲内の地面上の座標と、撮影画像上の座標との間には1対1の対応関係が存在する。従って、この対応関係を利用すれば、図3Bに例示した撮影画像を、図3Aに例示したように車両1の上方から地面を見下ろしたような画像(俯瞰画像)に変換することができる。これが、車載カメラ10F(あるいは車載カメラ10R)の撮影画像を視線変換して俯瞰画像を生成する際の考え方である。 Thus, there is a one-to-one correspondence between the coordinates on the ground within the shooting range of the in-vehicle camera 10F and the coordinates on the shot image. Therefore, by using this correspondence, the captured image illustrated in FIG. 3B can be converted into an image (overhead image) that looks down on the ground from above the vehicle 1 as illustrated in FIG. 3A. This is the way of thinking when generating a bird's-eye view image by converting the line-of-sight of a captured image of the in-vehicle camera 10F (or in-vehicle camera 10R).
 このような視線変換を行うことにより、図4Aに示した撮影画像は、図4Bに示すような俯瞰画像に変換される。こうして得られた俯瞰画像は、遠くの位置ほど左右方向に広がった画像となる。これは、車載カメラ10F(あるいは車載カメラ10R)の撮影範囲が左右両側に広がっていることによる(図3A参照)。図4Bでは、俯瞰画像中で左側に広がった領域を領域L、右側に広がった領域を領域Rと表示している。 By performing such line-of-sight conversion, the captured image shown in FIG. 4A is converted into an overhead image as shown in FIG. 4B. The bird's-eye view image thus obtained becomes an image that spreads in the left-right direction as the position is further away. This is because the imaging range of the in-vehicle camera 10F (or in-vehicle camera 10R) is widened on both the left and right sides (see FIG. 3A). In FIG. 4B, the region extending to the left in the overhead image is displayed as region L, and the region expanding to the right is displayed as region R.
 図4Bに示した状態では、領域Lも領域Rも、車載カメラ10Fの撮影領域内にある。しかし、車両1が前進すれば、領域Lは車両1の左方向の死角領域となり、領域Rは車両1の右方向の死角領域となる。 In the state shown in FIG. 4B, both the region L and the region R are within the imaging region of the in-vehicle camera 10F. However, if the vehicle 1 moves forward, the region L becomes a blind spot region in the left direction of the vehicle 1, and the region R becomes a blind spot region in the right direction of the vehicle 1.
 そこで、車載カメラ10Fの撮影画像を視線変換して得られた俯瞰画像(撮影俯瞰画像)を記憶しておき、車両1の移動に伴って、過去の撮影俯瞰画像の中から、現在の死角領域に対応する部分の画像(履歴画像)を切り出して表示してやる。こうすれば、車両1の左右方向の領域についても俯瞰画像を表示することが可能となる。 Therefore, a bird's-eye view image (shooting bird's-eye view image) obtained by line-of-sight conversion of the photographed image of the in-vehicle camera 10F is stored, and the current blind spot area is selected from the past photographed bird's-eye images as the vehicle 1 moves. The image (history image) of the part corresponding to is cut out and displayed. In this way, it is possible to display a bird's-eye view image also in the left and right area of the vehicle 1.
 図5には、時間の経過と共に車両1が移動し、それに伴って車両1の左右の領域に履歴画像が表示される様子が例示されている。以下、時間の経過に沿って説明すると、先ず始めに時刻Taでは、車両1の前後の撮影領域に、撮影俯瞰画像が表示されている。尚、この時点では、車両1の左右の死角領域には、まだ履歴画像が表示されていないものとしている。 FIG. 5 illustrates a state in which the vehicle 1 moves with time and a history image is displayed in the left and right areas of the vehicle 1 accordingly. Hereinafter, the description will be made along the passage of time. First, at time Ta, a photographing overhead image is displayed in the photographing regions before and after the vehicle 1. At this time, it is assumed that no history image has been displayed in the left and right blind spot areas of the vehicle 1.
 時刻Tbでは、車両1が少しだけ前進した結果、時刻Taでは車両1の前方に表示されていた撮影俯瞰画像の一部(車両1から見て手前側)が、死角領域に移動する。そこで、この死角領域については、時刻Taでの撮影俯瞰画像を切り出して履歴画像として表示する。図5の時刻Tbの画像中で、車両1の左右の死角領域に「a」と表示しているのは、この部分の画像が、時刻Taで得られた撮影俯瞰画像から切り出された履歴画像であることを表したものである。 At time Tb, as a result of the vehicle 1 moving forward a little, a part of the photographed bird's-eye view image displayed in front of the vehicle 1 at the time Ta (front side as viewed from the vehicle 1) moves to the blind spot area. Therefore, for this blind spot area, a captured bird's-eye view image at time Ta is cut out and displayed as a history image. In the image at time Tb in FIG. 5, “a” is displayed in the left and right blind spot areas of the vehicle 1 because this part of the image is a history image cut out from the photographed overhead image obtained at time Ta. It represents that.
 また、車両1の前後の撮影領域については、時刻Tbで車載カメラ10F,10Rが撮影した画像に基づいて新たな撮影俯瞰画像が生成されるので、この新たな撮影俯瞰画像を表示する。 In addition, for the shooting areas before and after the vehicle 1, a new shooting overhead image is generated on the basis of the images shot by the in- vehicle cameras 10F and 10R at time Tb, so this new shooting overhead image is displayed.
 時刻Tcでは、更に車両1が少しだけ前進した結果、時刻Tbで車両1の前方に表示されていた撮影俯瞰画像の一部が死角領域に移動する。そこで、この死角領域については、時刻Tbでの撮影俯瞰画像を切り出して履歴画像として表示する。また、時刻Tbで車両1の死角領域に表示されていた履歴画像も、車両1が前進した分だけ、車両1の後方に移動する。図5の時刻Tcの画像中で、車両1の左右の死角領域に「a」または「b」と表示しているのは、これらの部分の画像が、時刻Taまたは時刻Tbで得られた撮影俯瞰画像から切り出された履歴画像であることを表したものである。 At time Tc, as a result of the vehicle 1 further moving forward a little, a part of the photographed bird's-eye view image displayed in front of the vehicle 1 at time Tb moves to the blind spot area. Therefore, for this blind spot area, a captured bird's-eye view image at time Tb is cut out and displayed as a history image. Further, the history image displayed in the blind spot area of the vehicle 1 at the time Tb also moves to the rear of the vehicle 1 by the amount that the vehicle 1 has advanced. In the image at time Tc in FIG. 5, “a” or “b” is displayed in the left and right blind spot areas of the vehicle 1 because the images of these portions are taken at time Ta or time Tb. It represents that it is a history image cut out from an overhead image.
 また、車両1の前後の撮影領域については、時刻Tcで新たに生成した撮影俯瞰画像を表示する。 In addition, for the shooting areas before and after the vehicle 1, a shooting overhead image newly generated at time Tc is displayed.
 時刻Td、時刻Teについても同様に、車両1が前進した分だけ、車両1の前方に表示されていた撮影俯瞰画像の中から履歴画像を切り出して、車両1の左右の死角領域に表示する。また、既に死角領域に表示されていた履歴画像については、車両1が前進した分だけ、車両1の後方に移動させる。 Similarly, for the time Td and the time Te, the history image is cut out from the photographed bird's-eye view image displayed in front of the vehicle 1 by the amount of advance of the vehicle 1 and displayed in the left and right blind spot areas of the vehicle 1. Further, the history image that has already been displayed in the blind spot area is moved to the rear of the vehicle 1 by the amount that the vehicle 1 has advanced.
 このような操作を続けることにより、時刻Tfでは、車両1の死角領域全体に履歴画像が表示され、その結果、車両1の全周に亘って俯瞰画像を表示することが可能となる。 By continuing such an operation, at time Tf, the history image is displayed in the entire blind spot area of the vehicle 1, and as a result, the overhead image can be displayed over the entire circumference of the vehicle 1.
 もっとも、このようにして車両1の死角領域にも俯瞰画像を表示した場合、車両1の移動速度が大きくなると、死角領域に表示する俯瞰画像の画質が悪化するという問題がある。これは次のような理由による。 However, when the bird's-eye view image is also displayed in the blind spot area of the vehicle 1 in this way, there is a problem that the image quality of the bird's-eye view image displayed in the blind spot area deteriorates when the moving speed of the vehicle 1 increases. This is due to the following reason.
 図6には、車両1の移動速度が大きい場合に、履歴画像を用いて車両1の死角領域に俯瞰画像を表示する様子が例示されている。 FIG. 6 illustrates a state in which an overhead image is displayed in the blind spot area of the vehicle 1 using a history image when the moving speed of the vehicle 1 is high.
 図6においても、時刻Taでは、図5と同様に履歴画像は表示されていないものとする。時刻Tbでは、車両1の移動速度が大きいことに対応して、図5の場合よりも車両1が大きく前進しており、これに伴って、時刻Taで車両1の前方に表示されていた撮影俯瞰画像から切り出される履歴画像(図中で「a」と表示)の大きさも大きくなる。そして、時刻Tcでは、更に車両1が前進して、時刻Tbで前方に表示されていた撮影俯瞰画像から履歴画像(図中で「b」と表示)が切り出されて、車両1の死角領域に表示される。また、時刻Tbで死角領域に表示されていた履歴画像(図中で「a」と表示)は、車両1の後方に移動する。その結果、図6に示した例では、時刻Tcで車両1の全周の俯瞰画像を表示することが可能となる。 Also in FIG. 6, it is assumed that the history image is not displayed at the time Ta as in FIG. At time Tb, in response to the moving speed of the vehicle 1 being large, the vehicle 1 is moving forward more greatly than in the case of FIG. 5, and accordingly, the image displayed in front of the vehicle 1 at time Ta. The size of the history image cut out from the overhead image (displayed as “a” in the figure) also increases. At time Tc, the vehicle 1 further moves forward, and a history image (shown as “b” in the figure) is cut out from the photographed bird's-eye view image displayed forward at time Tb. Is displayed. In addition, the history image (displayed as “a” in the drawing) displayed in the blind spot area at time Tb moves to the rear of the vehicle 1. As a result, in the example shown in FIG. 6, it is possible to display an overhead view image of the entire circumference of the vehicle 1 at time Tc.
 ここで、時刻Tcで表示される俯瞰画像の中で、車両1の前方に表示されている撮影俯瞰画像(図中で「c」と表示)と、その撮影俯瞰画像に隣接する履歴画像(図中で「b」と表示)との境目部分に着目する。同様に、新しい履歴画像(図中で「b」と表示)と、古い履歴画像(図中で「a」と表示)との境目部分に着目する。 Here, among the bird's-eye view images displayed at time Tc, a shooting bird's-eye view image (displayed as “c” in the drawing) displayed in front of the vehicle 1 and a history image (see FIG. Attention is paid to the boundary portion between “b” and “. Similarly, attention is paid to a boundary portion between a new history image (displayed as “b” in the drawing) and an old history image (displayed as “a” in the drawing).
 図7には、これらの着目している境界部分が拡大して示されている。先ず始めに、図中で「c」と表示した撮影俯瞰画像と、図中で「b」と表示した履歴画像との境目部分に注目する。この境目に隣接する領域の撮影俯瞰画像(図中で細い破線で囲って表示)は、車両1から見て直ぐ近くの地面を撮影した画像を、視線変換して得られた画像である。また、境目に隣接する領域の履歴画像(図中で細い一点鎖線で囲って表示)は、車両1から見て遠くの地面を撮影した画像を、視線変換して得られた画像である。 FIG. 7 shows the boundary portion of interest in an enlarged manner. First, attention is paid to a boundary portion between a captured bird's-eye view image displayed as “c” in the drawing and a history image displayed as “b” in the drawing. The imaged bird's-eye view image of the area adjacent to this boundary (displayed by being surrounded by a thin broken line in the figure) is an image obtained by performing line-of-sight conversion on an image obtained by photographing the ground immediately near the vehicle 1. A history image of a region adjacent to the boundary (indicated by a thin dashed line in the drawing) is an image obtained by performing line-of-sight conversion on an image of the ground far away from the vehicle 1.
 そして、図4Aと図4Bに示したように、視線変換では、近くを写した部分の画像に比べて、遠くを写した部分の画像は大きく拡大される。このため、境目部分では、拡大率が大きく異なる2つの画像が繋ぎ合わされているように見えてしまい、画質を大きく悪化させることになる。 Then, as shown in FIGS. 4A and 4B, in the line-of-sight conversion, the image of the portion that is far away is greatly enlarged compared to the image of the portion that is close. For this reason, at the boundary part, it looks as if two images having greatly different enlargement ratios are connected, and the image quality is greatly deteriorated.
 図中で「b」と表示した新しい履歴画像と、図中で「a」と表示した古い履歴画像との境目部分についても、全く同様なことが当て嵌まる。すなわち、境目に隣接する部分の新しい履歴画像(図中で細い破線で囲って表示)は、車両1の近くの画像を視線変換した画像であり、これに対して、境目に隣接する部分の古い履歴画像(図中で細い一点鎖線で囲って表示)は、車両1から遠くの画像を視線変換した画像である。このため、この境目部分でも、拡大率が大きく異なる2つの画像が繋ぎ合わされているように見えてしまい、画質が大きく悪化する。 The same applies to the boundary between the new history image displayed as “b” in the figure and the old history image displayed as “a” in the figure. That is, a new history image adjacent to the boundary (indicated by a thin broken line in the figure) is an image obtained by line-of-sight conversion of an image near the vehicle 1, whereas an old part adjacent to the boundary is old. A history image (indicated by a thin dotted line in the figure) is an image obtained by line-of-sight conversion of an image far from the vehicle 1. For this reason, even at this boundary portion, it looks as if two images with greatly different enlargement rates are connected, and the image quality is greatly deteriorated.
 図6および図7では、説明の便宜から、画質が悪化する境目部分が2箇所であるものとしているが、実際には、より多くの境目部分が発生し、それらの箇所で画質が悪化する。その結果、車両1の死角領域に表示される俯瞰画像の画質が大きく悪化する。 6 and 7, for convenience of explanation, it is assumed that there are two boundary portions where the image quality deteriorates, but in reality, more boundary portions are generated, and the image quality deteriorates at those portions. As a result, the image quality of the overhead view image displayed in the blind spot area of the vehicle 1 is greatly deteriorated.
 もちろん、車両1の移動速度が大きくなるに従って、車載カメラ10F,10Rでの画像の撮影周期を短くすることにより、個々の履歴画像が大きくならないようにすれば、こうした画質の悪化を回避することができる。しかし、これでは、車載カメラ10F,10Rの撮影画像を視線変換して撮影俯瞰画像を生成する処理や、履歴画像を切り出して死角領域に表示する処理などを、より短い周期で実行する必要が生じるため、処理負担が増加してしまう。 Of course, if the history image is not enlarged by shortening the image capturing period of the in- vehicle cameras 10F and 10R as the moving speed of the vehicle 1 increases, such deterioration of the image quality can be avoided. it can. However, in this case, it is necessary to execute processing for generating a photographic overhead view image by converting the captured images of the in- vehicle cameras 10F and 10R, processing for cutting out a history image and displaying it in a blind spot area, etc. at a shorter cycle. Therefore, the processing burden increases.
 そこで、本実施例の俯瞰画像表示装置100は、次のような俯瞰画像表示処理を行う。
(第1実施例の俯瞰画像表示処理)
 図8および図9には、本実施例の俯瞰画像表示装置100が実行する第1実施例の俯瞰画像表示処理のフローチャートが示されている。この処理は、車両1のエンジンが起動した状態で、表示画面11に俯瞰画像を表示するための所定の操作が運転者によって行われると開始される。
Therefore, the overhead image display apparatus 100 according to the present embodiment performs the following overhead image display processing.
(Overhead image display processing of the first embodiment)
8 and 9 show a flowchart of the overhead image display process of the first embodiment executed by the overhead image display apparatus 100 of the present embodiment. This process is started when a predetermined operation for displaying an overhead image on the display screen 11 is performed by the driver while the engine of the vehicle 1 is activated.
 図示されるように、第1実施例の俯瞰画像表示処理(S100)を開始すると、先ず始めに車載カメラ10Fおよび車載カメラ10Rから、車両1の前方および後方の撮影画像を取得する(S101)。 As shown in the figure, when the bird's-eye view image display process (S100) of the first embodiment is started, first, captured images of the front and rear of the vehicle 1 are acquired from the in-vehicle camera 10F and the in-vehicle camera 10R (S101).
 そして、車載カメラ10F,10Rから取得した撮影画像を視線変換して、車両1の前方および後方の撮影領域についての撮影俯瞰画像を生成する(S102)。 Then, the captured images acquired from the in- vehicle cameras 10F and 10R are line-of-sight-converted, and the captured overhead images for the front and rear imaging areas of the vehicle 1 are generated (S102).
 その後、表示画面11上で車載カメラ10Fおよび車載カメラ10Rの撮影領域に対応するフレームメモリーに、前方および後方の撮影俯瞰画像を書き込む(S103)。ここで、フレームメモリーとは、表示画面11に画像を表示するために用いられるメモリー領域である。表示画面11に表示される画像はフレームメモリー上に生成される。そして、フレームメモリー上で画像が完成したら、フレームメモリーのデータを映像信号として出力することによって、表示画面11に画像が表示される。フレームメモリー上の1つ1つのアドレスは、表示画面11上の1つ1つの画素と対応している。S103では、表示画面11上で車載カメラ10Fの撮影領域に対応するフレームメモリーの各アドレスには前方の撮影俯瞰画像を書き込み、車載カメラ10Rの撮影領域に対応するフレームメモリーの各アドレスには後方の撮影俯瞰画像を書き込む。 Thereafter, the front and rear photographing overhead images are written in the frame memory corresponding to the photographing regions of the in-vehicle camera 10F and the in-vehicle camera 10R on the display screen 11 (S103). Here, the frame memory is a memory area used for displaying an image on the display screen 11. An image displayed on the display screen 11 is generated on the frame memory. When the image is completed on the frame memory, the image is displayed on the display screen 11 by outputting the data in the frame memory as a video signal. Each address on the frame memory corresponds to each pixel on the display screen 11. In S103, a front bird's-eye view image is written in each address of the frame memory corresponding to the shooting area of the in-vehicle camera 10F on the display screen 11, and a rear address is written in each address of the frame memory corresponding to the shooting area of the in-vehicle camera 10R. Write a bird's-eye view image.
 続いて、俯瞰画像表示装置100は、車両1の移動情報を取得する(S104)。ここで、車両1の移動情報とは、操舵角センサー12や、車速センサー13、シフト位置センサー14などから取得された情報である。 Subsequently, the overhead image display device 100 acquires movement information of the vehicle 1 (S104). Here, the movement information of the vehicle 1 is information acquired from the steering angle sensor 12, the vehicle speed sensor 13, the shift position sensor 14, and the like.
 そして、車速センサー13から取得した車両1の移動速度が、所定の表示速度以下であるか否かを判断する(S105)。ここで表示速度とは、車両1の死角領域に履歴画像を用いて俯瞰画像を表示するか否かを判断するための基準となる速度である。表示速度は、時速10km~20kmの速度範囲で、予め適切な速度に設定されている。 Then, it is determined whether or not the moving speed of the vehicle 1 acquired from the vehicle speed sensor 13 is equal to or lower than a predetermined display speed (S105). Here, the display speed is a reference speed for determining whether to display an overhead image using a history image in the blind spot area of the vehicle 1. The display speed is set to an appropriate speed in advance within a speed range of 10 km to 20 km per hour.
 前述したように、この俯瞰画像表示処理は、俯瞰画像を表示して車両1の周囲の状況を確認したいと考えた運転者が、所定の操作を行うことによって開始される。そして、運転者が俯瞰画像を用いて車両1の周囲の状況を確認したいと考えるのは、主に車両1の移動速度が小さい場合である。 As described above, this bird's-eye view image display process is started when a driver who wants to display a bird's-eye view image and check the situation around the vehicle 1 performs a predetermined operation. The driver wants to check the situation around the vehicle 1 using the bird's-eye view image mainly when the moving speed of the vehicle 1 is low.
 しかし、だからといって、俯瞰画像を表示するための操作が、車両1の移動速度が小さい状態で行われるとは限らない。例えば、狭い路地を通過したり、狭い駐車スペースに駐車したりする場合に、それら路地や駐車スペースが近付いてくると、車両1を減速させながら、予め俯瞰画像の表示を開始する場合もある。 However, this does not necessarily mean that the operation for displaying the overhead view image is performed in a state where the moving speed of the vehicle 1 is low. For example, when passing through a narrow alley or parking in a narrow parking space, when the alley or the parking space approaches, the vehicle 1 may be displayed in advance while the vehicle 1 is decelerated.
 そこで、第1実施例の俯瞰画像表示装置では、撮影俯瞰画像を生成してフレームメモリーに書き込むと(S102、S103)、車両1の移動速度が表示速度以下か否かを判断する(S105)。そして、車両1の表示速度が表示速度以下であった場合には(S105:yes)、履歴画像を生成するために、その撮影俯瞰画像を、撮影タイミングが識別可能な状態で(たとえば、タイムスタンプと共に)記憶する(S109)。 Therefore, in the bird's-eye view image display device according to the first embodiment, when the photographed bird's-eye view image is generated and written in the frame memory (S102, S103), it is determined whether or not the moving speed of the vehicle 1 is equal to or lower than the display speed (S105). When the display speed of the vehicle 1 is equal to or lower than the display speed (S105: yes), in order to generate a history image, the captured bird's-eye view image is displayed in a state where the shooting timing can be identified (for example, a time stamp). (S109)
 これに対して、車両1の表示速度が表示速度よりも大きかった場合には(S105:no)、車両1の移動速度が記憶速度以下であるか否かを判断する(S106)。ここで、記憶速度とは、履歴画像の生成に備えて撮影俯瞰画像を記憶するか否かの判断基準となる速度である。第1実施例では、表示速度よりも大きな適切な速度が、記憶速度として予め設定されている。尚、第1実施例では、記憶速度は表示速度よりも時速5km大きな値に設定されている。 On the other hand, when the display speed of the vehicle 1 is larger than the display speed (S105: no), it is determined whether or not the moving speed of the vehicle 1 is equal to or lower than the storage speed (S106). Here, the storage speed is a speed that is a criterion for determining whether or not to store a photographic overhead image in preparation for the generation of a history image. In the first embodiment, an appropriate speed larger than the display speed is preset as the storage speed. In the first embodiment, the storage speed is set to a value 5 km / h higher than the display speed.
 その結果、車両1の移動速度が記憶速度よりも小さかった場合は(S106:yes)、履歴画像の生成に備えて、撮影タイミングが識別可能な状態で(たとえば、タイムスタンプと共に)、撮影俯瞰画像を記憶する(S107)。 As a result, when the moving speed of the vehicle 1 is lower than the storage speed (S106: yes), the shooting overhead image is displayed in a state where the shooting timing is identifiable (for example, with a time stamp) in preparation for generation of a history image. Is stored (S107).
 続いて、車両1の移動速度が表示速度以下か否かを判断し(S108)、移動速度が未だ表示速度以下まで低下していない場合は(S108:no)、車両1の死角領域に履歴画像を表示する必要はないと判断できるので、フレームメモリー上の画像を表示画面11に出力する(図9のS114)。フレームメモリー上の撮影領域に対応するアドレスには、S103で撮影俯瞰画像が書き込まれているので、表示画面11には、車両1の前方および後方の撮影領域に俯瞰画像が表示される。 Subsequently, it is determined whether or not the moving speed of the vehicle 1 is equal to or lower than the display speed (S108). If the moving speed has not yet decreased below the display speed (S108: no), a history image is displayed in the blind spot area of the vehicle 1. Since it is determined that it is not necessary to display the image, the image on the frame memory is output to the display screen 11 (S114 in FIG. 9). Since the shooting bird's-eye view image is written in the address corresponding to the shooting area on the frame memory in S103, the bird's-eye view image is displayed on the display screen 11 in the shooting areas in front and behind the vehicle 1.
 これに対して、車両1の移動速度が表示速度以下と判断した場合は(S105:yes、あるいはS108:yes)、車両1の死角領域に履歴画像を表示するために、次のような処理を開始する。 On the other hand, when it is determined that the moving speed of the vehicle 1 is equal to or lower than the display speed (S105: yes or S108: yes), in order to display a history image in the blind spot area of the vehicle 1, the following processing is performed. Start.
 先ず、車両1が前進中か否かを判断する(図9のS110)。車両1が前進中か否かは、S104で取得した移動情報に基づいて判断することができる。 First, it is determined whether or not the vehicle 1 is moving forward (S110 in FIG. 9). Whether or not the vehicle 1 is moving forward can be determined based on the movement information acquired in S104.
 その結果、車両1が前進中であった場合は(S110:yes)、車両1の前方に表示した過去の撮影俯瞰画像の中から、車両1の死角領域に対応する画像(履歴画像)を切り出す(S111)。これに対して、車両1が後進中であった場合は(S110:no)、車両1の後方に表示した過去の撮影俯瞰画像の中から、車両1の死角領域に対応する画像(履歴画像)を切り出す(S112)。 As a result, when the vehicle 1 is moving forward (S110: yes), an image (history image) corresponding to the blind spot area of the vehicle 1 is cut out from the past shooting overhead images displayed in front of the vehicle 1. (S111). On the other hand, when the vehicle 1 is moving backward (S110: no), an image (history image) corresponding to the blind spot area of the vehicle 1 from the past shooting overhead images displayed behind the vehicle 1 Is cut out (S112).
 そして、得られた履歴画像を、表示画面11上で車載カメラ10Fおよび車載カメラ10Rの死角領域に対応するフレームメモリーに書き込む(S113)。 Then, the obtained history image is written in the frame memory corresponding to the blind spot area of the in-vehicle camera 10F and the in-vehicle camera 10R on the display screen 11 (S113).
 また、車両1の前方および後方の撮影俯瞰画像は、図8のS103で既にフレームメモリーに書き込まれているので、履歴画像が書き込まれることによって、フレームメモリー上では、表示画面11に表示する画像が完成する。 In addition, since the front and rear photographing overhead images of the vehicle 1 are already written in the frame memory in S103 of FIG. 8, the image displayed on the display screen 11 is displayed on the frame memory by writing the history image. Complete.
 そこで、俯瞰画像表示装置100は、フレームメモリー上の画像を表示画面11に出力する(S114)。その結果、表示画面11には、車両1の周辺を上方から見たような俯瞰画像が表示される。 Therefore, the overhead image display device 100 outputs the image on the frame memory to the display screen 11 (S114). As a result, a bird's-eye view image as if the periphery of the vehicle 1 was viewed from above is displayed on the display screen 11.
 以上のようにして、フレームメモリー上の画像を表示画面11に表示したら(S114)、車両1の周辺領域についての俯瞰画像の表示を終了するか否かを判断する(S115)。その結果、まだ表示を終了しない場合は(S115:no)、処理の先頭に戻って、再び、車載カメラ10F,10Rによって得られた撮影画像を取得した後(図8のS101)、上述した続く一連の処理を実行する。 As described above, when the image on the frame memory is displayed on the display screen 11 (S114), it is determined whether or not the display of the overhead image for the peripheral area of the vehicle 1 is terminated (S115). As a result, when the display is not finished yet (S115: no), the process returns to the top of the process, and again after acquiring the captured images obtained by the in- vehicle cameras 10F and 10R (S101 in FIG. 8), the above-described continues. A series of processing is executed.
 これに対して、表示を終了すると判断した場合は(図9のS115:yes)、第1実施例の俯瞰画像表示処理を終了する。 On the other hand, if it is determined that the display is to be terminated (S115 in FIG. 9: yes), the overhead image display process of the first embodiment is terminated.
 図10には、上述した第1実施例の俯瞰画像表示処理によって、表示画面11に俯瞰画像が表示される様子が例示されている。図示した例では、車両1の移動速度の減速中に、運転者によって、俯瞰画像の表示を開始する操作が行われたものとしている。 FIG. 10 illustrates a state in which an overhead image is displayed on the display screen 11 by the overhead image display processing of the first embodiment described above. In the illustrated example, it is assumed that an operation for starting the display of an overhead image is performed by the driver while the moving speed of the vehicle 1 is decelerated.
 俯瞰画像の表示を開始する操作が行われた時刻T1の時点では、車両1の移動速度が表示速度よりも高く、記憶速度よりも高くなっている。このため、前述した俯瞰画像表示処理では、移動速度が表示速度よりも高く(図8のS105:no)、且つ記憶速度よりも高い(S106:no)と判断されるので、車両1の左右の死角領域に履歴画像が表示されることがない。また、車両1の前後の撮影領域には、俯瞰画像の表示を開始する操作が運転者によって行われたことに対応して、撮影俯瞰画像が表示される。 The moving speed of the vehicle 1 is higher than the display speed and higher than the storage speed at the time T1 when the operation for starting the display of the overhead image is performed. For this reason, in the above-described bird's-eye view image display processing, it is determined that the moving speed is higher than the display speed (S105: no in FIG. 8) and higher than the storage speed (S106: no). No history image is displayed in the blind spot area. In addition, in the photographing areas before and after the vehicle 1, a photographed bird's-eye view image is displayed in response to an operation for starting display of the bird's-eye view image performed by the driver.
 その後、車両1の移動速度が表示速度まで低下すると、車両1の左右の死角領域に履歴画像が表示される。従って、車両1の前後の撮影領域に表示された撮影俯瞰画像と合わせて、車両1の全周に俯瞰画像が表示されることになる。 Thereafter, when the moving speed of the vehicle 1 decreases to the display speed, a history image is displayed in the left and right blind spots of the vehicle 1. Therefore, the bird's-eye view image is displayed on the entire circumference of the vehicle 1 together with the shooting bird's-eye view images displayed in the shooting areas before and after the vehicle 1.
 このように、第1実施例の俯瞰画像表示処理では、車両1の移動速度が表示速度よりも大きい段階で、運転者によって、俯瞰画像を表示するための操作が行われた場合でも、車両1の移動速度が表示速度以下になるまでは、履歴画像を用いて車両1の死角領域に俯瞰画像が表示されることがない。このため、図6および図7を用いて前述したような画質の悪い俯瞰画像が表示画面11に表示されることを回避することができる。 Thus, in the bird's-eye view image display process of the first embodiment, even when the driver performs an operation for displaying the bird's-eye view image at a stage where the moving speed of the vehicle 1 is higher than the display speed, the vehicle 1. The overhead image is not displayed in the blind spot area of the vehicle 1 using the history image until the moving speed becomes equal to or lower than the display speed. For this reason, it is possible to avoid the overhead image having poor image quality as described above with reference to FIGS. 6 and 7 being displayed on the display screen 11.
 もちろん、第1実施例の俯瞰画像表示処理では、運転者によって俯瞰画像の表示開始の操作が行われても、車両1の移動速度が表示速度以下になるまでは、車両1の死角領域の俯瞰画像が表示されなくなる。しかし、運転者が死角領域の俯瞰画像を本当に確認する必要が生じるのは、狭い路地を通過する際や、狭い駐車スペースに駐車する際のように、車両1の移動速度が小さい場合である。従って、移動速度が表示速度以下になったら履歴画像を表示するようにしておけば、表示速度よりも大きい場合には履歴画像を表示しなくても、実用上の問題が生じることはない。 Of course, in the bird's-eye view image display process of the first embodiment, even if the driver performs an operation to start displaying the bird's-eye view image, the bird's eye view of the blind spot area of the vehicle 1 is kept until the moving speed of the vehicle 1 becomes lower than the display speed. The image disappears. However, the driver actually needs to confirm the bird's eye view of the blind spot area when the moving speed of the vehicle 1 is low, such as when passing through a narrow alley or parking in a narrow parking space. Therefore, if the history image is displayed when the moving speed becomes lower than the display speed, there is no practical problem even if the history image is not displayed when the moving speed is higher than the display speed.
 また、上述した第1実施例の俯瞰画像表示処理では、車両1の移動速度が表示速度まで低下するよりも前に、記憶速度まで低下した時刻T2(図10参照)の時点で、撮影俯瞰画像の記憶が開始される。このため、車両1の移動速度が表示速度まで低下した時刻T3の時点では、履歴画像を切り出すための撮影俯瞰画像が既に記憶されている。その結果、車両1の移動速度が表示速度に達すると、車両1の死角領域全体の履歴画像を直ちに表示することが可能となる。 Further, in the above-described overhead image display processing of the first embodiment, the photographed overhead image is taken at time T2 (see FIG. 10) when the vehicle 1 is reduced to the storage speed before the moving speed is reduced to the display speed. Memory is started. For this reason, at the time T3 when the moving speed of the vehicle 1 is reduced to the display speed, a photographic overhead view image for cutting out the history image is already stored. As a result, when the moving speed of the vehicle 1 reaches the display speed, a history image of the entire blind spot area of the vehicle 1 can be displayed immediately.
 加えて、車両1の移動速度が記憶速度よりも大きい間は、撮影俯瞰画像を記憶しておく必要がないので、俯瞰画像表示装置100の処理負担を軽減することも可能となる。
(第1実施例の変形例)
 上述した第1実施例の俯瞰画像表示処理では、撮影俯瞰画像を記憶するための記憶速度は、適切な速度に予め設定されているものとして説明した。しかし、車両1の移動速度の減速度合(以下、減速度)に応じて、適切な記憶速度を設定してもよい。
In addition, while the moving speed of the vehicle 1 is higher than the storage speed, it is not necessary to store the captured bird's-eye view image, so that the processing burden on the bird's-eye view image display device 100 can be reduced.
(Modification of the first embodiment)
In the above-described bird's-eye view image display process of the first embodiment, it has been described that the storage speed for storing the captured bird's-eye view image is preset to an appropriate speed. However, an appropriate storage speed may be set according to the degree of deceleration of the moving speed of the vehicle 1 (hereinafter referred to as deceleration).
 以下では、このような変形例について、上述した第1実施例との相違点を中心に簡単に説明する。 Hereinafter, such a modified example will be briefly described focusing on differences from the first embodiment described above.
 図8を用いて前述したように、第1実施例の俯瞰画像表示処理では、車両1の移動速度が表示速度よりも大きいと判断すると(S105:no)、続いて、車両1の移動速度が記憶速度以下であるか否かを判断した(S106)。 As described above with reference to FIG. 8, in the overhead image display process of the first embodiment, if it is determined that the moving speed of the vehicle 1 is higher than the display speed (S105: no), then the moving speed of the vehicle 1 is It was determined whether or not the speed was lower than the storage speed (S106).
 しかし、第1実施例の変形例の俯瞰画像表示処理では、車両1の移動速度が表示速度よりも大きいと判断すると(S105:no)、図11に示すような処理を行う。 However, in the overhead image display process of the modification of the first embodiment, if it is determined that the moving speed of the vehicle 1 is higher than the display speed (S105: no), the process as shown in FIG. 11 is performed.
 先ず始めに、車両1が減速中か否かを判断する(S120)。減速中か否かは、車両1の移動情報に基づいて判断することができる。その結果、車両1が減速中でないと判断した場合は(S120:no)、車両1の移動速度は表示速度から遠ざかっていることになるので、撮影俯瞰画像を記憶することなく、図9のS114に移動して、フレームメモリー上の画像を表示画面11に向かって出力する。 First, it is determined whether or not the vehicle 1 is decelerating (S120). Whether or not the vehicle is decelerating can be determined based on movement information of the vehicle 1. As a result, when it is determined that the vehicle 1 is not decelerating (S120: no), the moving speed of the vehicle 1 is far from the display speed, so that the photographed bird's-eye view image is not stored and S114 in FIG. 9 is stored. To output the image on the frame memory toward the display screen 11.
 これに対して、車両1が減速中と判断した場合は(S120:yes)、車両1の減速度を算出する(S121)。ここでは、減速度として、単位時間あたりの車両1の移動速度の減少量を算出する。 On the other hand, when it is determined that the vehicle 1 is decelerating (S120: yes), the deceleration of the vehicle 1 is calculated (S121). Here, the amount of decrease in the moving speed of the vehicle 1 per unit time is calculated as the deceleration.
 続いて、算出した減速度に応じた記憶速度を取得する(S122)。減速度に対する記憶速度は、図12に示す計算式によって算出することができる。尚、計算式中の「a」は減速度を表し、「V1 」は記憶速度を表し、「VL」は車両1の車長を表している。もちろん、減速度に対する記憶速度の算出結果を予めテーブルとして記憶しておき、テーブルを参照することで、算出することなく、減速度に対する記憶速度を取得するようにしても良い。 Subsequently, the storage speed corresponding to the calculated deceleration is acquired (S122). The storage speed for the deceleration can be calculated by the calculation formula shown in FIG. In the calculation formula, “a” represents deceleration, “V1” represents storage speed, and “VL” represents vehicle length of the vehicle 1. Of course, the calculation result of the storage speed with respect to the deceleration may be stored in advance as a table, and the storage speed with respect to the deceleration may be acquired without referring to the table.
 尚、図12の計算式は、
Figure JPOXMLDOC01-appb-I000001
である。
The calculation formula of FIG.
Figure JPOXMLDOC01-appb-I000001
It is.
 その後、図8のS106に移動して、車両1の移動速度が記憶速度以下か否かを判断する。この時の判断に用いる記憶速度は、先にS122で取得した移動速度である。 Thereafter, the process moves to S106 in FIG. 8 to determine whether or not the moving speed of the vehicle 1 is equal to or lower than the storage speed. The storage speed used for the determination at this time is the movement speed previously acquired in S122.
 このようにすることで、車両1の移動速度が表示速度まで低下したちょうどその時に、車両1の死角領域全体の履歴画像を生成するための撮影俯瞰画像が、概ね過不足無く記憶されているような状態とすることができる。 By doing in this way, at the same time when the moving speed of the vehicle 1 is reduced to the display speed, a photographic overhead view image for generating a history image of the entire blind spot area of the vehicle 1 seems to be stored with almost no excess or deficiency. It can be in a state.
 こうしたことが可能となる理由について、図13を用いて説明する。例えば、車両1の減速中の時刻T1の時点で、運転者の操作によって俯瞰画像の表示が開始されたとする。その時の減速度で車両1が減速していくとすると、時刻T3で表示速度V0に達するので、それより前に撮影俯瞰画像の記憶を開始すればよい。尚、表示速度V0は予め決められている。 The reason why this is possible will be described with reference to FIG. For example, it is assumed that the overhead image display is started by the driver's operation at time T1 during deceleration of the vehicle 1. If the vehicle 1 decelerates at the deceleration at that time, the display speed V0 is reached at time T3, so that it is only necessary to start storing the captured overhead image before that. The display speed V0 is determined in advance.
 今、時刻T2で記憶を開始したとすると、時刻T2から時刻T3までの間に車両1が移動する距離は、図13中で斜線を付した面積となる。従って、この面積が、車両1の車長VLと等しくなるような時刻T2を選択しておけば、時刻T3では、ちょうど車長VLに相当する距離の撮影俯瞰画像が記憶されることになり、車両1の死角領域全体の履歴画像を生成することが可能となる。また、このように選択された時刻T2での速度が、記憶速度V1 となる。 Now, assuming that the storage is started at time T2, the distance that the vehicle 1 moves from time T2 to time T3 is an area hatched in FIG. Therefore, if a time T2 is selected such that this area is equal to the vehicle length VL of the vehicle 1, a photographed overhead image at a distance corresponding to the vehicle length VL is stored at the time T3. A history image of the entire blind spot area of the vehicle 1 can be generated. Further, the speed at time T2 selected in this way becomes the storage speed V1.
 時刻T2から時刻T3までの経過時間をT、車両1の減速度をaとすると、図13中で斜線を付した部分の面積は、V0・T + 1/2・a・T^2と表現される。
この値が車長VLに等しくなるとき、次式が成立する。
If the elapsed time from time T2 to time T3 is T and the deceleration of the vehicle 1 is a, the hatched area in FIG. 13 is expressed as V0 · T + 1/2 · a · T ^ 2. Is done.
When this value becomes equal to the vehicle length VL, the following equation is established.
 VL = V0・T + 1/2・a・T^2
これを解くことによって、次式が得られる。
VL = V0 ・ T + 1/2 ・ a ・ T ^ 2
By solving this, the following equation is obtained.
 T = (-V0 + (2・VL・a+V0^2)^0.5)/a
その結果、記憶速度V1 は、
 V1 = V0 + a・T
 V1 = (2・VL・a + V0^2)^0.5によって算出することができる。
T = (-V0 + (2 ・ VL ・ a + V0 ^ 2) ^ 0.5) / a
As a result, the storage speed V1 is
V1 = V0 + a · T
V1 = (2 · VL · a + V0 ^ 2) ^ 0.5 can be calculated.
 上述した第1実施例の変形例の俯瞰画像表示処理では、車両1の減速度に応じた記憶速度で撮影俯瞰画像の記憶を開始することができるので、車両1が表示速度まで低下した時点では、ちょうど車長に相当する距離の撮影俯瞰画像を記憶しておくことができる。その結果、車両1の死角領域で履歴画像の表示を開始する際には、死角領域中に履歴画像が表示されない領域が発生することを抑制することができるので、運転者が車両1の周辺を容易に確認することが可能となる。 In the overhead image display process of the modified example of the first embodiment described above, since the storage of the captured overhead image can be started at the storage speed corresponding to the deceleration of the vehicle 1, when the vehicle 1 decreases to the display speed. It is possible to store a photographed bird's-eye view image at a distance corresponding to the vehicle length. As a result, when the display of the history image is started in the blind spot area of the vehicle 1, it is possible to suppress the occurrence of an area in which the history image is not displayed in the blind spot area. It can be easily confirmed.
 また、必要以上に速い時点から撮影俯瞰画像の記憶を開始することもないので、俯瞰画像表示装置100の処理負担が増加することも回避することが可能となる。
(第2実施例の俯瞰画像表示処理)
 上述した第1実施例、および第1実施例の変形例では、車両1の移動速度が表示速度に達したら、車両1の死角領域に履歴画像を用いた俯瞰画像を表示するものとして説明した。また、死角領域での俯瞰画像の表示を開始する際には、始めから死角領域全体で俯瞰画像を表示することができるように、車両1の移動速度が記憶速度まで低下したら撮影俯瞰画像の記憶を開始するものとして説明した。
Further, since the storage of the captured bird's-eye view image is not started at a point faster than necessary, it is possible to avoid an increase in the processing load on the bird's-eye view image display device 100.
(Overhead image display processing of the second embodiment)
In the first embodiment described above and the modification of the first embodiment, it has been described that an overhead image using a history image is displayed in the blind spot area of the vehicle 1 when the moving speed of the vehicle 1 reaches the display speed. In addition, when the display of the overhead view image in the blind spot area is started, when the moving speed of the vehicle 1 decreases to the storage speed so that the overhead view image can be displayed in the entire blind spot area, the captured overhead image is stored. Explained as starting.
 しかし、車両1の死角領域での俯瞰画像の表示を開始するタイミングは、必ずしも、車両1の移動速度が表示速度まで低下したタイミングと正確に一致している必要はない。すなわち、表示速度から大きく外れるのでなければ、車両1の死角領域での俯瞰画像の表示を、表示速度より大きな速度から開始しても、あるいは表示速度よりも小さな速度から開始しても構わない。 However, the timing for starting the display of the bird's-eye view image in the blind spot area of the vehicle 1 does not necessarily need to exactly coincide with the timing at which the moving speed of the vehicle 1 is reduced to the display speed. That is, as long as it does not deviate significantly from the display speed, the display of the bird's-eye view image in the blind spot area of the vehicle 1 may be started from a speed higher than the display speed or a speed lower than the display speed.
 図14および図15には、第2実施例の俯瞰画像表示処理のフローチャートが示されている。前述した第1実施例あるいは第1実施例の変形例の俯瞰画像表示処理では、撮影俯瞰画像の記憶を開始すると、車両1の移動速度が表示速度まで低下したことを確認して、履歴画像を用いた俯瞰画像の表示を開始した。これに対して、以下に説明する第2実施例の俯瞰画像表示処理(S200)は、車両1の移動速度が表示速度まで低下したことではなく、撮影俯瞰画像の記憶を開始してから車両1の車長に相当する距離だけ移動したことを確認して、履歴画像を用いた俯瞰画像の表示を開始する点が大きく異なっている。以下、フローチャートを参照しながら、第2実施例の俯瞰画像表示処理(S200)について簡単に説明する。尚、第2実施例の俯瞰画像表示処理も、前述した第1実施例の俯瞰画像表示処理と同様に、車両1のエンジンが起動した状態で、表示画面11に俯瞰画像を表示するための所定の操作が運転者によって行われると、俯瞰画像表示装置100によって開始される。 14 and 15 show a flowchart of the overhead image display process of the second embodiment. In the above-described overhead image display processing of the first embodiment or the modification of the first embodiment, when storage of the captured overhead image is started, it is confirmed that the moving speed of the vehicle 1 has decreased to the display speed, and the history image is displayed. Display of the overhead image used was started. On the other hand, the overhead image display process (S200) of the second embodiment described below is not that the moving speed of the vehicle 1 has decreased to the display speed, but that the vehicle 1 has been stored after the storage of the captured overhead image has started. After confirming that the vehicle has moved by a distance corresponding to the vehicle length, the display of the overhead image using the history image is greatly different. Hereinafter, the overhead image display processing (S200) of the second embodiment will be briefly described with reference to the flowchart. Note that the overhead image display processing of the second embodiment is also a predetermined for displaying the overhead image on the display screen 11 in a state where the engine of the vehicle 1 is activated, similarly to the overhead image display processing of the first embodiment described above. When the operation is performed by the driver, the overhead image display device 100 starts.
 図14に示されるように、第2実施例の俯瞰画像表示処理(S200)においても、先ず始めに車載カメラ10Fおよび車載カメラ10Rから撮影画像を取得する(S201)。そして、取得した撮影画像を視線変換して撮影俯瞰画像を生成し(S202)、撮影タイミングが識別可能な状態で(たとえば、タイムスタンプと共に)メモリーに記憶する(S203)。 As shown in FIG. 14, also in the bird's-eye view image display process (S200) of the second embodiment, first, a captured image is acquired from the in-vehicle camera 10F and the in-vehicle camera 10R (S201). Then, the acquired captured image is transformed into a line of sight to generate a captured overhead image (S202), and stored in the memory (with a time stamp, for example) in a state where the imaging timing can be identified (S203).
 その後、表示画面11上で車載カメラ10Fおよび車載カメラ10Rの撮影領域に対応するフレームメモリーに、前方および後方の撮影俯瞰画像を書き込んだ後(S203)、車両1の移動速度を含む移動情報を取得する(S204)。 Then, after writing the front and rear shooting overhead images on the display screen 11 in the frame memory corresponding to the shooting area of the in-vehicle camera 10F and the in-vehicle camera 10R (S203), the movement information including the moving speed of the vehicle 1 is acquired. (S204).
 そして、車両1の移動速度が、表示速度以下であるか否かを判断する(S205)。その結果、車両1の移動速度が、表示速度以下であった場合は(S205:yes)、履歴画像を生成するために撮影俯瞰画像を記憶する(S209)。 Then, it is determined whether or not the moving speed of the vehicle 1 is equal to or lower than the display speed (S205). As a result, when the moving speed of the vehicle 1 is equal to or lower than the display speed (S205: yes), a captured overhead image is stored to generate a history image (S209).
 これに対して、車両1の移動速度が表示速度よりも大きかった場合には(S205:no)、車両1が減速中か否かを判断する(S206)。その結果、車両1が減速中でなかった場合は(S206:no)、車両1の移動速度が表示速度から遠ざかっているので、撮影俯瞰画像を記憶する必要はないと判断して、フレームメモリー上に書き込まれた画像を表示画面11に出力する(図15のS215)。フレームメモリー上の撮影領域に対応するアドレスには、S203で撮影俯瞰画像が書き込まれているので、表示画面11には、車両1の前方および後方の撮影領域に俯瞰画像が表示される。 On the other hand, when the moving speed of the vehicle 1 is larger than the display speed (S205: no), it is determined whether or not the vehicle 1 is decelerating (S206). As a result, when the vehicle 1 is not decelerating (S206: no), it is determined that there is no need to store the captured bird's-eye view image because the moving speed of the vehicle 1 is far from the display speed. Is output to the display screen 11 (S215 in FIG. 15). Since the shooting bird's-eye view image is written in the address corresponding to the shooting area on the frame memory in S203, the bird's-eye view image is displayed on the display screen 11 in the shooting areas in front and rear of the vehicle 1.
 これに対して、車両1が減速中であった場合は(S206:yes)、車両1の移動速度が記憶速度以下であるか否かを判断する(S207)。ここで、第2実施例では、記憶速度は予め設定されているものとする。もちろん、第1実施例の変形例と同様な方法を用いて、車両1の減速度に応じた記憶速度を設定しても良い。 On the other hand, when the vehicle 1 is decelerating (S206: yes), it is determined whether or not the moving speed of the vehicle 1 is equal to or lower than the storage speed (S207). Here, in the second embodiment, the storage speed is set in advance. Of course, you may set the memory | storage speed according to the deceleration of the vehicle 1 using the method similar to the modification of 1st Example.
 その結果、車両1の移動速度が記憶速度よりも小さかった場合は(S207:yes)、履歴画像の生成に備えて撮影俯瞰画像を記憶する(S208)。 As a result, when the moving speed of the vehicle 1 is lower than the storage speed (S207: yes), the photographed overhead image is stored in preparation for the generation of the history image (S208).
 その後、第2実施例の俯瞰画像表示処理では、撮影俯瞰画像の記憶を開始してから、車両1の車長に相当する距離を移動したか否かを判断する(S210)。その結果、まだ車長に相当する距離を移動していないと判断した場合は(S210:no)、フレームメモリー上に書き込まれた撮影俯瞰画像を表示画面11に出力する(図15のS215)。 Thereafter, in the bird's-eye view image display process of the second embodiment, it is determined whether or not the distance corresponding to the vehicle length of the vehicle 1 has moved since the storage of the photographed bird's-eye view image is started (S210). As a result, if it is determined that the distance corresponding to the vehicle length has not yet been moved (S210: no), the captured overhead image written in the frame memory is output to the display screen 11 (S215 in FIG. 15).
 これに対して、車長に相当する距離を移動したと判断した場合は(S210:yes)、車両1が前進中か否かを判断する(図15のS211)。そして、車両1が前進中であった場合は(S211:yes)、車両1の前方に表示した過去の撮影俯瞰画像の中から、車両1の死角領域に対応する画像(履歴画像)を切り出す(S212)。これに対して、車両1が後進中であった場合は(S211:no)、車両1の後方に表示した過去の撮影俯瞰画像の中から、車両1の死角領域に対応する画像(履歴画像)を切り出す(S213)。 On the other hand, when it is determined that the distance corresponding to the vehicle length has been moved (S210: yes), it is determined whether or not the vehicle 1 is moving forward (S211 in FIG. 15). If the vehicle 1 is moving forward (S211: yes), an image (history image) corresponding to the blind spot area of the vehicle 1 is cut out from the past shooting overhead image displayed in front of the vehicle 1 ( S212). On the other hand, when the vehicle 1 is moving backward (S211: no), an image (history image) corresponding to the blind spot area of the vehicle 1 from the past bird's-eye view images displayed behind the vehicle 1 is displayed. Is cut out (S213).
 そして、得られた履歴画像を、表示画面11上で車載カメラ10Fおよび車載カメラ10Rの死角領域に対応するフレームメモリーに書き込んだ後(S214)、フレームメモリー上の画像を表示画面11に出力する(S215)。 Then, after the obtained history image is written in the frame memory corresponding to the blind spot area of the in-vehicle camera 10F and the in-vehicle camera 10R on the display screen 11 (S214), the image in the frame memory is output to the display screen 11 ( S215).
 車両1の前方および後方の撮影俯瞰画像は、図14のS203で既にフレームメモリーに書き込まれているので、こうして履歴画像を書き込んだ後にフレームメモリー上の画像を表示画面11に出力することで、表示画面11には、車両1の周辺を上方から見たような俯瞰画像が表示される。 Since the front and rear photographing overhead images of the vehicle 1 are already written in the frame memory in S203 of FIG. 14, the history image is written in this way, and the image on the frame memory is output to the display screen 11 to display the image. The screen 11 displays an overhead image as if the periphery of the vehicle 1 was viewed from above.
 その後、車両1の周辺領域についての俯瞰画像の表示を終了するか否かを判断する(S216)。その結果、まだ表示を終了しない場合は(S216:no)、処理の先頭に戻って、再び、車載カメラ10F,10Rによって得られた撮影画像を取得した後(図14のS201)、上述した続く一連の処理を実行する。 Thereafter, it is determined whether or not to end the display of the bird's-eye view image for the peripheral area of the vehicle 1 (S216). As a result, if the display is not finished yet (S216: no), the process returns to the top of the process, and the captured images obtained by the in- vehicle cameras 10F and 10R are acquired again (S201 in FIG. 14), and then continue as described above. A series of processing is executed.
 これに対して、表示を終了すると判断した場合は(S216:yes)、第2実施例の俯瞰画像表示処理を終了する。 On the other hand, when it is determined that the display is to be ended (S216: yes), the overhead image display process of the second embodiment is ended.
 以上に説明した第2実施例の俯瞰画像表示処理においては、車両1の移動速度が記憶速度まで低下したら、撮影俯瞰画像の記憶を開始して、撮影俯瞰画像を記憶しながら車両1の車長に相当する距離を移動したら、履歴画像を用いた俯瞰画像の表示を開始する。 In the overhead view image display process of the second embodiment described above, when the moving speed of the vehicle 1 decreases to the storage speed, storage of the captured overhead image is started and the vehicle length of the vehicle 1 is stored while storing the captured overhead image. When the distance corresponding to is moved, the display of the overhead image using the history image is started.
 このため、車両1の移動速度が大きい場合には、履歴画像を用いた俯瞰画像が車両1の死角領域に表示されないので、図6および図7を用いて前述したような画質の悪い俯瞰画像が表示画面11に表示されることを回避することができる。 For this reason, when the moving speed of the vehicle 1 is high, the overhead image using the history image is not displayed in the blind spot area of the vehicle 1, so that the overhead image with poor image quality as described above with reference to FIGS. Display on the display screen 11 can be avoided.
 また、上述した第2実施例の俯瞰画像表示処理では、車両1の車長に相当する距離の撮影俯瞰画像が記憶されたら、履歴画像を用いた俯瞰画像の表示を開始するので、始めから車両1の死角領域全体の俯瞰画像を表示することが可能となる。 Further, in the above-described overhead image display processing of the second embodiment, when a shooting overhead image at a distance corresponding to the vehicle length of the vehicle 1 is stored, the display of the overhead image using the history image is started. It is possible to display a bird's-eye view image of the entire one blind spot area.
 本開示の一例に係る車両周辺画像表示装置および車両周辺画像表示方法は、車載カメラによって得られた撮影画像に視線変換を施して撮影俯瞰画像を生成する。そして、車両の移動速度を含む移動情報を取得して、移動速度が所定の表示速度より大きい場合には、表示画面上で、車載カメラが撮影した前記周辺領域に対応する位置に撮影俯瞰画像を表示する。また、移動速度が所定の表示速度以下の場合には、表示画面上で、車載カメラの死角領域に対応する位置に、履歴画像を表示する。 The vehicle periphery image display device and the vehicle periphery image display method according to an example of the present disclosure perform a line-of-sight conversion on a captured image obtained by an in-vehicle camera to generate a captured overhead image. Then, when the movement information including the movement speed of the vehicle is acquired and the movement speed is larger than the predetermined display speed, a captured overhead image is displayed on the display screen at a position corresponding to the peripheral area captured by the in-vehicle camera. indicate. When the moving speed is equal to or lower than the predetermined display speed, the history image is displayed on the display screen at a position corresponding to the blind spot area of the in-vehicle camera.
 本開示の車両周辺画像表示装置および車両周辺画像表示方法によれば、車両の移動速度が表示速度よりも大きい場合には履歴画像が表示されないので、画質の悪化した俯瞰画像が表示されて運転者に違和感を与えることを回避できる。 According to the vehicle periphery image display device and the vehicle periphery image display method of the present disclosure, since the history image is not displayed when the moving speed of the vehicle is higher than the display speed, an overhead image with deteriorated image quality is displayed and the driver is displayed. It can avoid giving a sense of incongruity.
 ここで、この出願に記載されるフローチャート、あるいは、フローチャートの処理は、複数のステップ(あるいはセクションと言及される)から構成され、各ステップは、たとえば、S100と表現される。さらに、各ステップは、複数のサブステップに分割されることができる、一方、複数のステップが合わさって一つのステップにすることも可能である。 Here, the flowchart described in this application or the processing of the flowchart is configured by a plurality of steps (or referred to as sections), and each step is expressed as S100, for example. Further, each step can be divided into a plurality of sub-steps, while a plurality of steps can be combined into one step.
 以上、各種の実施例および変形例について説明したが、本開示は上記の実施例および変形例に限られるものではなく、その要旨を逸脱しない範囲において種々の態様で実施することができる。 Although various embodiments and modifications have been described above, the present disclosure is not limited to the above-described embodiments and modifications, and can be implemented in various modes without departing from the scope of the present disclosure.

Claims (5)

  1.  車載カメラ(10F,10R)を用いて車両(1)の周辺領域を撮影した画像を表示画面(11)上に表示する車両周辺画像表示装置(100)であって、
     前記車載カメラ(10F,10R)によって得られた撮影画像に対して、該車載カメラ(10F,10R)に写った前記周辺領域を前記車両(1)の上方から撮影した画像に変換する視線変換を施すことによって、撮影俯瞰画像を生成する撮影俯瞰画像生成部(102)と、
     前記撮影俯瞰画像を記憶する撮影俯瞰画像記憶部(106)と、
     前記表示画面(11)上で、前記車載カメラ(10F,10R)が撮影した前記周辺領域に対応する位置に前記撮影俯瞰画像を表示する撮影俯瞰画像表示部(103)と、
     前記車両(1)の移動速度を含む移動情報を取得する移動情報取得部(104)と、
     前記車両(1)の周辺領域の中で前記車載カメラ(10F,10R)の死角となる死角領域については、前記車両(1)が現在の位置に達する前に生成された前記撮影俯瞰画像の中から該死角領域を撮影した部分の画像である履歴画像を、前記移動情報に基づいて切り出して、前記表示画面(11)上の該死角領域に対応する位置に表示する履歴画像表示部(103)と
     を備え、
     前記履歴画像表示部(103)は、前記移動速度が所定の表示速度を超えている場合には前記履歴画像を表示せず、該表示速度以下の場合に該履歴画像を表示する
     車両周辺画像表示装置。
    A vehicle peripheral image display device (100) for displaying an image obtained by photographing a peripheral region of a vehicle (1) using a vehicle-mounted camera (10F, 10R) on a display screen (11),
    Line-of-sight conversion is performed on the captured image obtained by the in-vehicle camera (10F, 10R) to convert the peripheral area captured by the in-vehicle camera (10F, 10R) into an image captured from above the vehicle (1). A shooting bird's-eye view image generation unit (102) that generates a shooting bird's-eye view image,
    A shooting overhead image storage unit (106) for storing the shooting overhead image;
    On the display screen (11), a shooting overhead image display unit (103) for displaying the shooting overhead image at a position corresponding to the peripheral area captured by the in-vehicle camera (10F, 10R);
    A movement information acquisition unit (104) for acquiring movement information including the movement speed of the vehicle (1);
    Regarding the blind spot area that is the blind spot of the in-vehicle camera (10F, 10R) in the peripheral area of the vehicle (1), the blind spot area image generated before the vehicle (1) reaches the current position A history image, which is an image of a portion of the blind spot area captured from the image, is cut out based on the movement information and displayed at a position corresponding to the blind spot area on the display screen (11). And
    The history image display unit (103) does not display the history image when the moving speed exceeds a predetermined display speed, and displays the history image when the moving speed is lower than the display speed. apparatus.
  2.  請求項1に記載の車両周辺画像表示装置であって、
     前記撮影俯瞰画像記憶部(106)は、前記移動速度が、前記表示速度より大きい所定の記憶速度を超えている場合には前記撮影俯瞰画像を記憶せず、該記憶速度以下の場合に該撮影俯瞰画像を記憶する
     車両周辺画像表示装置。
    The vehicle periphery image display device according to claim 1,
    The shooting bird's-eye image storage unit (106) does not store the shooting bird's-eye view image when the moving speed exceeds a predetermined storage speed larger than the display speed, and takes the shooting when the moving speed is lower than the storage speed. A vehicle peripheral image display device that stores a bird's-eye view image.
  3.  請求項2に記載の車両周辺画像表示装置であって、
     前記履歴画像表示部(103)は、前記移動速度が前記表示速度以下になると、前記死角領域の全ての領域が前記撮影俯瞰画像記憶部(106)に記憶されているか否かを判断して、記憶されていると判断した場合に、前記履歴画像を表示する
     車両周辺画像表示装置。
    The vehicle periphery image display device according to claim 2,
    The history image display unit (103) determines whether or not all areas of the blind spot area are stored in the shooting overhead image storage unit (106) when the moving speed is equal to or lower than the display speed. A vehicle periphery image display device that displays the history image when it is determined that it is stored.
  4.  請求項2または請求項3に記載の車両周辺画像表示装置であって、
     前記移動速度が前記表示速度より大きく且つ該移動速度が減少している場合に、該移動速度の減速度合を検出して、該減速度合に基づいて前記記憶速度を決定する記憶速度決定部(105)を備える
     車両周辺画像表示装置。
    The vehicle periphery image display device according to claim 2 or 3, wherein
    When the moving speed is larger than the display speed and the moving speed is decreasing, a storage speed determining unit (105) detects a degree of deceleration of the moving speed and determines the storage speed based on the degree of deceleration. A vehicle periphery image display device.
  5.  車載カメラ(10F,10R)を用いて車両(1)の周辺領域を撮影した画像を表示画面(11)上に表示する車両周辺画像表示方法(S100)であって、
     前記車載カメラによって得られた撮影画像に対して、該車載カメラに写った前記周辺領域を前記車両(1)の上方から撮影した画像に変換する視線変換を施すことによって、撮影俯瞰画像を生成する撮影俯瞰画像生成工程(S102)と、
     前記撮影俯瞰画像を記憶する撮影俯瞰画像記憶工程(S107、S109)と、
     前記表示画面(11)上で、前記車載カメラが撮影した前記周辺領域に対応する位置に前記撮影俯瞰画像を表示する撮影俯瞰画像表示工程(S103、S114)と、
     前記車両(1)の移動速度を含む移動情報を取得する移動情報取得工程(S104)と、
     前記車両(1)の周辺領域の中で前記車載カメラの死角となる死角領域については、前記車両(1)が現在の位置に達する前に生成された前記撮影俯瞰画像の中から該死角領域を撮影した部分の画像である履歴画像を、前記移動情報に基づいて切り出して、前記表示画面(11)上の該死角領域に対応する位置に表示する履歴画像表示工程(S113、S114)と
     を備え、
     前記履歴画像表示工程(S113、S114)は、前記移動速度が所定の表示速度を超えている場合には前記履歴画像を表示せず、該表示速度以下の場合に該履歴画像を表示する
     車両周辺画像表示方法。
    A vehicle peripheral image display method (S100) for displaying an image obtained by photographing a peripheral region of a vehicle (1) using a vehicle-mounted camera (10F, 10R) on a display screen (11),
    A photographic overhead view image is generated by subjecting a captured image obtained by the in-vehicle camera to line-of-sight conversion for converting the peripheral region captured by the in-vehicle camera into an image captured from above the vehicle (1). A shooting overhead image generation step (S102);
    A shooting overhead image storage step (S107, S109) for storing the shooting overhead image;
    On the display screen (11), a shooting overhead image display step (S103, S114) for displaying the shooting overhead image at a position corresponding to the peripheral area captured by the in-vehicle camera;
    A movement information acquisition step (S104) for acquiring movement information including the movement speed of the vehicle (1);
    Regarding the blind spot area that is the blind spot of the in-vehicle camera in the peripheral area of the vehicle (1), the blind spot area is selected from the photographed bird's-eye view image generated before the vehicle (1) reaches the current position. A history image display step (S113, S114) that cuts out a history image that is an image of a photographed part based on the movement information and displays it at a position corresponding to the blind spot area on the display screen (11). ,
    The history image display step (S113, S114) does not display the history image when the moving speed exceeds a predetermined display speed, and displays the history image when the moving speed is lower than the display speed. Image display method.
PCT/JP2015/000746 2014-03-05 2015-02-18 Vehicle peripheral image display device and method for displaying vehicle peripheral image WO2015133072A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
DE112015001088.2T DE112015001088T5 (en) 2014-03-05 2015-02-18 Vehicle environment image display device and vehicle environment image display method
CN201580011579.4A CN106063258B (en) 2014-03-05 2015-02-18 Vehicle-surroundings image display device, vehicle-surroundings image display method

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2014042987A JP6326869B2 (en) 2014-03-05 2014-03-05 Vehicle periphery image display device and vehicle periphery image display method
JP2014-042987 2014-03-05

Publications (1)

Publication Number Publication Date
WO2015133072A1 true WO2015133072A1 (en) 2015-09-11

Family

ID=54054892

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2015/000746 WO2015133072A1 (en) 2014-03-05 2015-02-18 Vehicle peripheral image display device and method for displaying vehicle peripheral image

Country Status (4)

Country Link
JP (1) JP6326869B2 (en)
CN (1) CN106063258B (en)
DE (1) DE112015001088T5 (en)
WO (1) WO2015133072A1 (en)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP6512145B2 (en) * 2016-03-22 2019-05-15 株式会社デンソー IMAGE PROCESSING APPARATUS, IMAGE PROCESSING METHOD, AND PROGRAM
KR101861523B1 (en) * 2016-06-17 2018-07-05 주식회사 와이즈오토모티브 Apparatus and method for supporting driving of vehicle
CN108068696B (en) * 2016-11-17 2020-06-19 比亚迪股份有限公司 Display method and display device of panoramic image
KR20180069380A (en) * 2016-12-15 2018-06-25 주식회사 와이즈오토모티브 Apparatus and method for supporting driving of vehicle
DE102017130169A1 (en) * 2017-02-24 2018-08-30 Denso Ten Limited CHARGING DEVICE SUPPORT
JP7318258B2 (en) * 2019-03-26 2023-08-01 コベルコ建機株式会社 Remote control system and remote control server

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2007148761A (en) * 2005-11-28 2007-06-14 Denso Corp Parking support system
JP2007288282A (en) * 2006-04-12 2007-11-01 Toyota Motor Corp Vehicle periphery monitoring device
JP2011022952A (en) * 2009-07-21 2011-02-03 Aisin Aw Co Ltd Drive assist device, drive assist method, and drive assist program
JP2013055410A (en) * 2011-09-01 2013-03-21 Denso Corp Vehicle peripheral image display control device

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4156214B2 (en) * 2001-06-13 2008-09-24 株式会社デンソー Vehicle periphery image processing apparatus and recording medium
JP3778849B2 (en) * 2001-12-18 2006-05-24 株式会社デンソー Vehicle periphery image processing apparatus and recording medium
JP4914458B2 (en) * 2009-02-12 2012-04-11 株式会社日本自動車部品総合研究所 Vehicle periphery display device
JP5035284B2 (en) * 2009-03-25 2012-09-26 株式会社日本自動車部品総合研究所 Vehicle periphery display device
US9418556B2 (en) * 2010-12-30 2016-08-16 Wise Automotive Corporation Apparatus and method for displaying a blind spot
JP5722127B2 (en) * 2011-06-07 2015-05-20 株式会社小松製作所 Work vehicle perimeter monitoring device

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2007148761A (en) * 2005-11-28 2007-06-14 Denso Corp Parking support system
JP2007288282A (en) * 2006-04-12 2007-11-01 Toyota Motor Corp Vehicle periphery monitoring device
JP2011022952A (en) * 2009-07-21 2011-02-03 Aisin Aw Co Ltd Drive assist device, drive assist method, and drive assist program
JP2013055410A (en) * 2011-09-01 2013-03-21 Denso Corp Vehicle peripheral image display control device

Also Published As

Publication number Publication date
CN106063258B (en) 2018-12-25
DE112015001088T5 (en) 2016-12-01
JP6326869B2 (en) 2018-05-23
CN106063258A (en) 2016-10-26
JP2015170915A (en) 2015-09-28

Similar Documents

Publication Publication Date Title
WO2015133072A1 (en) Vehicle peripheral image display device and method for displaying vehicle peripheral image
WO2016002163A1 (en) Image display device and image display method
JP5262976B2 (en) In-vehicle display device
JP6586849B2 (en) Information display device and information display method
JP2007274377A (en) Periphery monitoring apparatus, and program
JP6425991B2 (en) Towing vehicle surrounding image generating apparatus and method for generating towing vehicle surrounding image
CN110877572A (en) Periphery monitoring device
JP2006287892A (en) Driving support system
JP6375633B2 (en) Vehicle periphery image display device and vehicle periphery image display method
JP2018144526A (en) Periphery monitoring device
JP2012040883A (en) Device for generating image of surroundings of vehicle
JP5262515B2 (en) Vehicle display device and display method
JP2016149613A (en) Camera parameter adjustment device
WO2013103115A1 (en) Image display device
JP7000383B2 (en) Image processing device and image processing method
JP2015171106A (en) Vehicle peripheral image display device and vehicle peripheral image display method
JP6327115B2 (en) Vehicle periphery image display device and vehicle periphery image display method
WO2016047037A1 (en) Vehicular image-processing apparatus
JP6407596B2 (en) Image processing apparatus and driving support system
JP4857159B2 (en) Vehicle driving support device
KR101861523B1 (en) Apparatus and method for supporting driving of vehicle
JP4945315B2 (en) Driving support system and vehicle
JP6788447B2 (en) Video output system
WO2021131481A1 (en) Display device, display method, and display program
JP4526868B2 (en) Top-view image generation apparatus and top-view image display method

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 15758342

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 112015001088

Country of ref document: DE

122 Ep: pct application non-entry in european phase

Ref document number: 15758342

Country of ref document: EP

Kind code of ref document: A1