WO2015122124A1

WO2015122124A1 - Vehicle periphery image display apparatus and vehicle periphery image display method

Info

Publication number: WO2015122124A1
Application number: PCT/JP2015/000279
Authority: WO
Inventors: 圭史冨山; 祥吾大宮; 典史松川; 純一北尾
Original assignee: 株式会社デンソー
Priority date: 2014-02-12
Filing date: 2015-01-22
Publication date: 2015-08-20
Also published as: JP2015154125A; DE112015000763T5; CN106031166A; JP6375633B2; CN106031166B

Abstract

A display apparatus for displaying a vehicle periphery image line-of-vision-converts a captured image of a vehicle-mounted camera to generate a captured-from-above image, and stores the generated captured-from-above image. If any movable body exists in the captured-from-above image, the display apparatus also stores the position of the movable body in the captured-from-above image. The display apparatus acquires vehicle movement information, cuts, out of the stored captured-from-above image, a part thereof that is a history image obtained by capturing the image of a vehicle blind spot area, and displays, on a display screen, the history image together with the captured-from-above image. At this moment, if the movable body exists in the history image, then the display apparatus displays the movable body in a different display mode from the display mode in the case where the movable body exists in the captured-from-above image.

Description

Vehicle periphery image display device and vehicle periphery image display method

Cross-reference of related applications

This disclosure is based on Japanese Patent Application No. 2014-24013 filed on February 12, 2014, the contents of which are incorporated herein.

This disclosure relates to a display device (Display Apparatus) and a display method (Display Method) for displaying an image obtained by photographing a peripheral region of a vehicle using an in-vehicle camera on a display screen.

An apparatus for displaying captured images obtained by vehicle-mounted cameras mounted on the front, rear, left, and right sides of the vehicle, by converting the line of sight into an image as if the vehicle was viewed from above, and displaying the obtained image (overhead image) It has been known. If the driver of the vehicle can confirm the bird's-eye view image, the positional relationship between the obstacle present in the surroundings and the host vehicle on which the device is mounted, the positional relationship between the display drawn on the road surface and the host vehicle, etc. It is possible to easily grasp the feeling.

In addition, a technique for displaying an overhead image of a peripheral region in the front-rear and left-right directions of the vehicle has been proposed (Patent Document 1) even though the vehicle-mounted cameras are not mounted on the left and right sides of the vehicle. With this technology, it is not possible to directly capture images in the left and right peripheral areas of the vehicle with the in-vehicle camera, but from the images of the in-vehicle cameras mounted on the front and rear of the vehicle, the overhead view image in the left and right direction is obtained according to the following principle. Generate.

First, in the captured image obtained by the in-vehicle camera in front of the vehicle, not only the area in front of the vehicle but also the diagonally forward areas are shown on both sides. Therefore, the bird's-eye view image obtained from the front vehicle-mounted camera also includes an image of a region diagonally forward in the left and right directions of the vehicle. This diagonally forward area eventually becomes an area in the left-right direction of the vehicle as the vehicle moves forward. Therefore, if an image of a region diagonally forward of the vehicle included in the overhead image is stored, it can be used as an overhead image in the left-right direction of the vehicle after the vehicle has advanced.

When the vehicle further moves forward, the image of the corresponding part may be cut out and used from the overhead view image stored before the forward movement. In this way, it is possible to display a bird's-eye view of the surrounding area in the front-rear and left-right directions of the vehicle without mounting on-vehicle cameras on the left and right sides of the vehicle.

Note that the bird's-eye view image displayed in the left-right direction of the vehicle is a past bird's-eye view image that was actually taken a little while ago. Therefore, an image cut out from a past bird's-eye view image and displayed in the left-right direction of the vehicle is called a “history image”.

Further, when the vehicle is moving backward, it is only necessary to store a bird's-eye view image by an in-vehicle camera behind the vehicle and display a history image cut out from the bird's-eye view image in the left and right direction portion of the vehicle. By so doing, it is possible to display a bird's-eye view image in the front-rear and left-right directions of the vehicle even when the vehicle is traveling backward.

JP 2002-373327 A

However, as described above, in the technology that uses a history image to display a bird's-eye view image including an area that is a blind spot of an in-vehicle camera, the driver misunderstands when a moving object exists in the display area. May give.

This is due to the following reasons. First, when the moving body is within the shooting range of the on-vehicle camera, the positional relationship between the host vehicle and the moving body and the positional relationship change by changing the line of sight of the image of the on-vehicle camera and displaying the overhead image. It is possible to correctly display the state of going. After that, when the moving body enters the blind spot of the in-vehicle camera as the vehicle moves, the overhead image obtained by converting the line of sight of the in-vehicle camera cannot display the moving body. Need arises. However, since the history image is an image taken a while ago, the correct position of the moving body is not always displayed. In addition, the position of the moving object displayed using the history image gradually moves in the opposite direction to the traveling direction of the vehicle as the vehicle progresses, but moves in the opposite direction by the amount of movement of the vehicle. Because it only does, it will appear as if the moving body has stopped against the ground. For this reason, even if the moving body is approaching to the immediate vicinity of the vehicle, the moving body is stopped, so that it may be misunderstood by the driver unless it contacts the vehicle.

This disclosure has been made in view of the above-described problems of the prior art, and even when a moving body exists around the vehicle, an overview of the surroundings of the vehicle using a history image without misleading the driver. An object is to provide a technique capable of displaying an image.

In order to solve the above-described problem, according to one example of the present disclosure, a vehicle peripheral image display device and a vehicle peripheral image display method are provided as follows. A photographic overhead view image is generated by performing line-of-sight conversion on a captured image obtained by the in-vehicle camera, and the obtained photographic overhead view image is stored. Further, when a moving body is present in the photographic overhead image, the position of the moving body in the photographic overhead image is also stored. Then, movement information relating to the amount of movement of the vehicle is acquired, and a history image, which is an image of a portion where the blind spot area of the vehicle is photographed, is cut out from the stored photographing bird's-eye view image and displayed together with the photographing bird's-eye view image Display above. At this time, if a moving object exists in the history image, the moving object is displayed in a display mode different from that in the case where the moving object exists in the captured overhead image.

In this way, since the display mode of the moving body on the display screen is changed, the driver can recognize that the moving body is displayed by the history image. As a result, the driver tries to confirm the moving body by, for example, direct visual observation, so that misunderstanding of the position of the moving body can be avoided.

The above and other objects, features and advantages of the present disclosure will become more apparent from the following detailed description with reference to the accompanying drawings.
Explanatory drawing which shows the vehicle carrying the image display apparatus of embodiment of this indication, An explanatory view showing a rough internal configuration of the image display device, Explanatory drawing which showed the picked-up image obtained by image | photographing the area | region ahead of a vehicle, Explanatory drawing about the principle of converting a shot image into a bird's-eye view image, Comparison explanatory diagram about the reason why there is a possibility of misleading the driver when there is a moving body around the vehicle, The flowchart figure of the first half part of the image display process of this embodiment, The flowchart figure of the latter half part of the image display process of this embodiment, Explanatory drawing which illustrated the history image obtained by the image display device of this embodiment, Explanatory drawing which illustrated the other aspect of the log | history image obtained by the image display apparatus of this embodiment.

Hereinafter, embodiments of the present disclosure will be described in order to clarify the contents of the present disclosure described above.

A. Device configuration :
FIG. 1 shows a vehicle 1 (also referred to as a host vehicle) equipped with an image display device 100. In the present embodiment, the image display device 100 is also referred to as a vehicle periphery image display device.

As illustrated, the vehicle 1 includes an in-vehicle camera 10F mounted in front of the vehicle 1, an in-vehicle camera 10R mounted in the rear of the vehicle 1, a display screen 11 mounted in the vehicle interior, and an image display device 100. Etc. In the present embodiment, the vehicle-mounted

cameras

10F and 10R are described as being mounted on the front and rear of the vehicle 1, respectively. However, the vehicle 1 mounted with either the vehicle-mounted camera 10F or the vehicle-mounted camera 10R is described. Also, the present disclosure can be applied.

When the image display device 100 receives an image of the periphery of the vehicle 1 taken by the in-

vehicle cameras

10F and 10R, the image display device 100 converts the image into an overhead view image taken from above the vehicle 1, and then displays the obtained image. It is displayed on the screen 11.

The vehicle 1 is also equipped with a steering angle sensor 12 for detecting the steering angle of the steering handle 2, a vehicle speed sensor 13 for detecting the vehicle speed, a shift position sensor 14 for detecting a shift position of a transmission (not shown), and the like. Yes. The outputs of the steering angle sensor 12, the vehicle speed sensor 13, and the shift position sensor 14 are input to the image display device 100, and the image display device 100 determines the movement amount (movement speed and movement direction) of the vehicle 1 based on these outputs. ) Related movement information.

And the image display apparatus 100 is based on the movement information of the vehicle 1 and the bird's-eye view images obtained by the in-

vehicle cameras

10F and 10R. A bird's-eye view image for (region) is generated and displayed on the display screen 11.

FIG. 2 shows a rough internal configuration of the image display apparatus 100 of the present embodiment. As shown in the figure, the image display apparatus 100 of the present embodiment includes a captured image acquisition unit 101, a captured overhead image generation unit 102, a moving body detection unit 103, a captured overhead image storage unit 104, and a movement information acquisition unit 105. And a display image generation unit 106.

Note that these six “sections” are abstractions in which the interior of the image display apparatus 100 is classified for convenience, focusing on the function of the image display apparatus 100 displaying an image around the vehicle 1 on the display screen 11. This does not represent that the image display device 100 is physically divided into six parts. Therefore, these “units” can be realized as software components as computer programs executed by the CPU, or can be realized as hardware components as electronic circuits including LSIs and memories. Further, it can be realized by combining these.

The captured image acquisition unit 101 is connected to the in-vehicle camera 10F and the in-vehicle camera 10R, and acquires a captured image obtained by capturing an area in front of the vehicle 1 from obliquely above from the in-vehicle camera 10F at a constant cycle (about 30 Hz). In addition, from the in-vehicle camera 10 </ b> R, a captured image obtained by capturing an area behind the vehicle 1 from obliquely above is acquired at a constant period (about 30 Hz).

The photographed bird's-eye view image generation unit 102 receives the images taken by the in-

vehicle cameras

10F and 10R from the photographed image acquisition unit 101 and generates an overhead image. That is, the photographed image obtained by the in-vehicle camera 10F is an image obtained by photographing an area in front of the vehicle 1 from obliquely above, and the photographed image is obtained from above the vehicle 1 by performing line-of-sight conversion described later on this image. Such a bird's-eye view image is generated. Similarly, the image captured by the in-vehicle camera 10 </ b> R is an image obtained by capturing an area behind the vehicle 1 from obliquely above, and this image is converted into an overhead view image captured from above the vehicle 1. A method for generating a bird's-eye view image by performing line-of-sight conversion on images captured by the in-

vehicle cameras

10F and 10R will be described later. In the present embodiment, the photographing overhead image generation unit 102 is also referred to as a photographing overhead image generation device / means.

As will be described later, in the image display device 100 according to the present embodiment, an overhead image is generated not only in the front-rear direction area of the vehicle 1 but also in the left-right direction area of the vehicle 1 and displayed on the display screen 11. be able to. Here, the bird's-eye view image displayed in the front-rear direction of the vehicle 1 is an image obtained by line-of-sight conversion of the current image captured by the in-

vehicle cameras

10F and 10R. On the other hand, the bird's-eye view image displayed in the left-right direction of the vehicle 1 is an image synthesized from the past bird's-eye view images displayed in the front-rear direction of the vehicle 1 based on the movement information of the vehicle 1.

Therefore, hereinafter, the bird's-eye view image displayed in the front-rear direction of the vehicle 1 is referred to as a “shooting bird's-eye view image” in order to distinguish it from the synthesized bird's-eye view image displayed in the left-right direction of the vehicle 1. The term “overhead image” simply refers to an image viewed from above the vehicle 1, and accordingly, both the imaged bird's-eye view image and the history image correspond to the “overhead image”. The shooting bird's-eye view image generation unit 102 outputs the generated shooting bird's-eye view image to the moving object detection unit 103, the shooting bird's-eye view image storage unit 104, and the display image generation unit 106.

The moving body detection unit 103 analyzes the shooting overhead image supplied from the shooting overhead image generation unit 102 and detects a moving body in the shooting overhead image. The moving object can be detected by comparing an image received in the past with a newly received image.

In the present embodiment, the moving body detection unit 103 is described as detecting a moving body from the captured bird's-eye view image. However, as indicated by the dashed arrows in the drawing, the captured image acquisition unit 101 receives the in-vehicle camera 10F, A moving image in the captured image may be detected by receiving a captured image by 10R. The moving body detection unit 103 is also referred to as a moving body detection device / means.

The photographic overhead image storage unit 104 stores the photographic overhead image supplied from the photographic overhead image generation unit 102. Further, when a moving body is detected by the moving body detection unit 103 in the captured bird's-eye view image, the position where the moving body is detected in the captured bird's-eye view image is also stored in association with the captured bird's-eye view image. In the present embodiment, the photographing overhead image storage unit 104 is also referred to as a photographing overhead image storage device / means.

Note that when the moving body detection unit 103 detects a moving body using an image (photographed images by the in-

vehicle cameras

10F and 10R) before the line-of-sight of the captured overhead image is not a captured overhead image, the captured image is captured. By performing line-of-sight conversion on the position of the moving body detected in the position, the position of the moving body in the captured bird's-eye view image can be obtained.

The movement information acquisition unit 105 is connected to the steering angle sensor 12, the vehicle speed sensor 13, and the shift position sensor 14. The movement information acquisition unit 105 acquires movement information of the vehicle 1 based on the outputs of these sensors. Here, the movement information of the vehicle 1 is information such as whether the vehicle 1 is moving forward, backward, or stopped, whether the vehicle 1 is moving straight, or whether it is turning right Whether the vehicle is turning left, or if it is turning, information on the magnitude of the turn, information on the moving speed of the vehicle 1, and the like. The movement information acquisition unit 105 outputs the acquired movement information of the vehicle 1 to the display image generation unit 106. The movement information acquisition unit 105 is also referred to as movement information acquisition device / means.

The display image generation unit 106 uses the shooting overhead image from the shooting overhead image generation unit 102, the movement information from the movement information acquisition unit 105, and the past shooting overhead image stored in the shooting overhead image storage unit 104. Thus, a bird's-eye view image of the peripheral region over the entire circumference of the vehicle 1 is generated and displayed on the display screen 11.

Here, the front and rear areas of the vehicle 1 are the imaging areas of the in-

vehicle cameras

10F and 10R, and therefore, an overhead image (captured overhead image) obtained by line-of-sight conversion of the images captured by the in-

vehicle cameras

10F and 10R may be displayed. However, since the areas in the right and left directions of the vehicle 1 are blind spots of the in-

vehicle cameras

10F and 10R, a bird's-eye view image cannot be obtained simply by performing line-of-sight conversion on the images captured by the in-

vehicle cameras

10F and 10R. Therefore, by cutting out an image (history image) of the corresponding part based on the movement information of the vehicle 1 from the past shooting bird's-eye images stored in the shooting bird's-eye image storage unit 104, An overhead image for the region is generated. In the present embodiment, the display image generation unit 106 is also referred to as a shooting overhead image display unit / device / means or a history image display unit / device / means.

Hereinafter, a method for generating a bird's-eye view image in an area in the left-right direction of the vehicle 1 will be described. As a preparation for this, first, a photographed bird's-eye view image is generated by performing line-of-sight conversion on images taken by the in-

vehicle cameras

10F and 10R. A method will be described. Then, based on this description, a method for generating a bird's-eye view image in an area in the left-right direction of the vehicle 1 will be described.

B. Principle of displaying a bird's-eye view of the surroundings of a vehicle using history images:
Consider a case where a plurality of landmarks are drawn in a grid pattern at equal intervals on the ground imaged by the in-vehicle camera 10F (or in-vehicle camera 10R) of the vehicle 1.

FIG. 3A shows a plurality of round marks drawn in a lattice pattern at equal intervals on the ground in front of the vehicle 1 as viewed from above the vehicle 1. In the figure, two alternate long and short dash lines extending obliquely from the in-vehicle camera 10F represent the boundary between the imaging region and the blind spot region of the in-vehicle camera 10F.

In addition, a round mark displayed with a thick solid line indicates that the mark is within the imaging region of the in-vehicle camera 10F, and a round mark displayed with a thin broken line indicates that the mark is mounted on the vehicle. It represents that it is in the blind spot area of the camera 10F. The in-vehicle camera 10F (or the in-vehicle camera 10R) images such a ground from obliquely above.

FIG. 3B shows a captured image obtained by the in-vehicle camera 10F. The photographed image shows a mark on the ground that exists within the photographing range of the in-vehicle camera 10F. For example, the mark a on the photographed image is the mark A on the ground in FIG. 3A, and the mark b on the photographed image is the mark B on the ground. Similarly, the mark c and the mark d on the photographed image are the marks C and D on the ground.

Thus, there is a one-to-one correspondence between the coordinates on the ground within the shooting range of the in-vehicle camera 10F and the coordinates on the shot image. Therefore, if this correspondence is used, the captured image illustrated in FIG. 3B is converted into an image (overhead image) in which the ground is looked down from above the vehicle 1 as illustrated in FIG. can do. This is the way of thinking when generating a bird's-eye view image by converting the line-of-sight of a captured image of the in-vehicle camera 10F (or in-vehicle camera 10R).

By performing such line-of-sight conversion, the captured image shown in FIG. 4A is converted into an overhead image as shown in FIG. The bird's-eye view image thus obtained becomes an image that spreads in the left-right direction as the position is further away. This is because the imaging range of the in-vehicle camera 10F (or in-vehicle camera 10R) is widened on both the left and right sides (see FIG. 3A). In FIG. 4B, a region that extends to the left in the overhead image is displayed as region L, and a region that extends to the right is displayed as region R.

In the state shown in FIG. 4B, both the region L and the region R are in the imaging region of the in-vehicle camera 10F. However, if the vehicle 1 moves forward, the region L becomes a blind spot region in the left direction of the vehicle 1, and the region R becomes a blind spot region in the right direction of the vehicle 1.

Therefore, a bird's-eye view image (shooting bird's-eye view image) obtained by line-of-sight conversion of the photographed image of the in-vehicle camera 10F is stored, and the current blind spot area is selected from the past photographed bird's-eye images as the vehicle 1 moves. The image (history image) of the part corresponding to is cut out and displayed. In this way, it is possible to display a bird's-eye view image also in the left and right area of the vehicle 1. The above is the basic concept of displaying a bird's-eye view image of the left and right region of the vehicle 1 using the history image.

However, when the bird's-eye view image around the vehicle 1 is displayed using the history image in this way, there is a possibility that the driver may be misunderstood if a moving object exists within the display range. The reason for this will be described with reference to FIG.

FIG. 5 shows a state in which the bird's-eye view around the vehicle 1 displayed on the display screen 11 changes as the vehicle 1 moves forward. In FIG. 5, the front portion of the vehicle 1 is enlarged and displayed.

Further, for convenience of understanding, two white lines are drawn on the ground in front of the vehicle 1 in parallel to the traveling direction of the vehicle 1, and one line is perpendicular to the traveling direction of the vehicle 1 on the back side. The white line is drawn. Further, here, it is assumed that a photographed bird's-eye view image generated by line-of-sight conversion in the past has not yet been stored.

FIG. 5A shows an overhead view image around the vehicle 1 displayed on the display screen 11 first. As shown in the drawing, in the imaging area in front of the vehicle 1, a shooting overhead image generated by line-of-sight conversion of a captured image of the in-vehicle camera 10 </ b> F is displayed. In this photographed bird's-eye view image, a moving object is shown on the white line. Further, as described above, since the photographed bird's-eye view image generated in the past is not stored, no history image is displayed in the left and right blind spot areas of the vehicle 1.

FIG. 5B shows an overhead view image around the vehicle 1 displayed on the display screen 11 with the vehicle 1 slightly advanced. In the bird's-eye view image displayed in the shooting area in front of the vehicle 1, the position of the white line is slightly closer to the vehicle 1 as the vehicle 1 slightly moves forward. Further, in FIG. 5A, the moving body that was on the white line has moved to the near side of the white line.

As described above, since the captured bird's-eye view image is an image obtained by line-of-sight conversion of the current image photographed by the in-vehicle camera 10F, the fact that the moving body is in front of the white line in the photographed bird's-eye view image is also the photographed image of the in-vehicle camera 10F. It can be considered that the moving body is in front of the white line. Therefore, on the display screen 11 shown in FIG. 5B, it can be considered that the position where the moving object actually exists is correctly displayed.

Further, in the display screen 11 shown in FIG. 5B, the left and right blind spot areas of the vehicle 1 are images (corresponding to images) of the corresponding part from the past photographing bird's-eye images displayed in FIG. Image) is cut out and displayed. In FIG. 5B to FIG. 5D below, the history image is displayed with diagonal lines in order to distinguish the captured overhead image from the history image.

FIG. 5C shows a bird's-eye view of the surroundings of the vehicle 1 displayed on the display screen 11 in a state where the vehicle 1 has further advanced a little. As the vehicle 1 moves forward, a white line ahead of the vehicle 1 is displayed at a position closer to the vehicle 1. As a result, the white line in the imaging area in front of the vehicle 1 is a slight part, and most of the white line exists in the left and right blind spot areas of the vehicle 1. However, as for the blind spot area, since a history image obtained by cutting out the corresponding portion from the past shooting overhead image displayed in FIG. 5A or 5B is displayed, You can confirm the position of the white line with.

Here, the moving object is also displayed on the display screen 11 shown in FIG. As is clear from the comparison between the moving body and the white line, the position of the moving body shown in FIG. 5C is not changed from the position of the moving body shown in FIG. This is because the moving body image in FIG. 5C is a history image cut out from the image displayed in the photographic overhead image in FIG. 5B, and the image in FIG. 5C is displayed. It is because it is a past image from the point of view.

Of course, it is considered that the moving body is actually moving further from the time when the image of FIG. 5B is displayed. If the moving direction and moving speed of the moving body do not change, the moving body at the time when the image of FIG. 5C is displayed should be present at the position of the white circle indicated by the broken line in the figure. . That is, in this case, the driver misunderstands that the position of the moving body is farther than the actual position.

FIG. 5D shows a bird's-eye view image around the vehicle 1 displayed on the display screen 11 in a state where the vehicle 1 has advanced a little further from the state shown in FIG. 5C. As shown in the drawing, in the image of FIG. 5D, the entire white line is displayed as a history image.

Further, a moving object is also displayed in the image of FIG. 5D, but since this moving object is displayed as a history image, the position of the moving object with respect to the white line is displayed in the image of FIG. 5B. It has not changed from where it was. However, if the moving direction and moving speed of the moving body do not change, the moving body should actually exist at the position of the white circle indicated by the broken line in the figure.

In addition, when the moving body is a pedestrian or the like, the moving direction and moving speed of the moving body are not always constant. Therefore, it is difficult to estimate the actual position of the moving body.

Thus, when displaying the image of the blind spot area around the vehicle 1 using the history image, if the mobile object enters the blind spot area, the correct position of the mobile object cannot be displayed, and the driver of the vehicle 1 May be misleading. Therefore, in order to display an image around the vehicle 1 using a history image without misleading the driver even when a moving object is present, the image display device 100 of the present embodiment has the following: Execute image display processing.

C. Image display processing of this embodiment:
6 and 7 show flowcharts of image display processing executed by the image display apparatus 100 of the present embodiment.

Here, the flowchart described in this application or the process of the flowchart includes a plurality of sections (or referred to as steps), and each section is expressed as S100, for example. Further, each section can be divided into a plurality of subsections, while a plurality of sections can be combined into one section. Further, each section can be referred to as a device, module, or means. In addition, each of the above sections or a combination thereof includes not only (i) a section of software combined with a hardware unit (eg, a computer), but also (ii) hardware (eg, an integrated circuit, As a section of (wiring logic circuit), it can be realized with or without the function of related devices. Furthermore, the hardware section can be included inside the microcomputer.

As shown in the figure, when the image display process is started, first, captured images of the front and rear of the vehicle 1 are acquired from the in-vehicle camera 10F and the in-vehicle camera 10R (S100). Then, the captured images acquired from the in-

vehicle cameras

10F and 10R are line-of-sight-converted to generate shooting overhead images for the shooting areas in front and rear of the vehicle 1 (S101).

Subsequently, a moving body is detected from the overhead images taken from the front and rear of the vehicle 1 (S102). Any known method may be used to detect the moving object. And it is judged whether the mobile body was detected (S103). As a result, when the moving object is not detected (S103: NO), the front and rear shooting overhead images are stored in the memory in a state where the shooting timing can be identified (for example, together with the time stamp) (S104). . On the other hand, when the moving body is detected (S103: YES), the photographing overhead image and the position of the moving body in the photographing overhead image are stored in the memory in a state where the photographing timing can be identified ( S105).

For example, if a moving object is detected in the front shooting bird's-eye image and a moving object is not detected in the rear shooting bird's-eye image, the position of the shooting bird's-eye image and the moving object is determined for the front shooting bird's-eye image. It memorize | stores (S105) and memorize | stores a photography bird's-eye view image about a back photography bird's-eye view image (S104).

Thereafter, the front and rear photographing overhead images are written in the frame memory corresponding to the photographing areas of the in-vehicle camera 10F and the in-vehicle camera 10R on the display screen 11 (S106). Here, the frame memory is a memory area used for displaying an image on the display screen 11. An image displayed on the display screen 11 is generated on the frame memory. When the image is completed on the frame memory, the image is displayed on the display screen 11 by outputting the data in the frame memory as a video signal.

Each address on the frame memory corresponds to each pixel on the display screen 11. In S106, the front bird's-eye view image is written in each address of the frame memory corresponding to the shooting area of the in-vehicle camera 10F on the display screen 11, and the rear address is written in each address of the frame memory corresponding to the shooting area of the in-vehicle camera 10R. Write a bird's-eye view image.

Subsequently, the image display device 100 acquires movement information (also referred to as movement data) of the vehicle 1 (S107). Here, the movement information of the vehicle 1 is information (or data) acquired from the steering angle sensor 12, the vehicle speed sensor 13, the shift position sensor 14, or the like.

Based on the acquired movement information, it is determined whether or not the vehicle 1 is moving (S108 in FIG. 7). If the vehicle 1 is moving (S108: YES), it is determined whether or not the vehicle 1 is moving forward. Is determined (S109). As a result, when the vehicle 1 is moving forward (S109: YES), an image (history image) corresponding to the blind spot area of the vehicle 1 is cut out from the past shooting overhead image displayed in front of the vehicle 1. (S110).

On the other hand, when the vehicle 1 is moving backward (S109: NO), an image (history image) corresponding to the blind spot area of the vehicle 1 from the past shooting overhead images displayed behind the vehicle 1 is displayed. Is cut out (S111). Since the movement information of the vehicle 1 has already been acquired in S107, it is possible to determine a portion corresponding to the current blind spot area of the vehicle 1 from the past photographed bird's-eye view images.

Subsequently, it is determined whether or not a moving object is present in the cut-out history image (S112). As described above, in the image display processing of the present embodiment, when a photographic bird's-eye view image is stored, it is determined whether or not a moving object exists in the photographic bird's-eye view image (S103 in FIG. 6). If it exists, the position of the moving body is also stored (S105). For this reason, it can be easily determined whether or not there is a moving object in the history image cut out from the photographed bird's-eye view image.

As a result, if a moving object is present in the history image (S112 in FIG. 7: YES), the display mode of the moving object in the history image is changed to a special mode different from the normal mode (S113). For example, the color of the moving object in the history image is changed (for example, changed to red) and displayed, the moving object is displayed surrounded by a red frame or a white frame, and the moving object image is displayed in advance. It replaces the stored graphic image and displays it. On the other hand, when there is no moving object in the history image (S112: NO), the operation for changing the display mode of the moving object is not performed.

Then, the history image obtained in this way is written in the frame memory corresponding to the blind spot area of the in-vehicle camera 10F and the in-vehicle camera 10R on the display screen 11 (S114). When there is a moving object in the history image, the history image changed to the special mode is written.

Also, since the bird's-eye view images of the front and rear of the vehicle 1 have already been written in the frame memory in S106 in FIG. 6, the image displayed on the display screen 11 is displayed on the frame memory by writing the history image. Complete.

Therefore, the image display device 100 outputs the image on the frame memory to the display screen 11 (S115). As a result, a bird's-eye view image as if the periphery of the vehicle 1 was viewed from above is displayed on the display screen 11.

In the foregoing, a series of processes (S109 to S114) for writing a history image to the frame memory corresponding to the blind spot area of the display screen 11 when the vehicle 1 is determined to be moving (S108: YES) has been described.

On the other hand, when it is determined that the vehicle 1 is stopped (S108: NO), the series of processing for writing the history image (S109 to S114) is not performed, and the image on the frame memory is output to the display screen 11. (S115). As a result, the display screen 11 displays an image in which only the captured bird's-eye images displayed before and after the vehicle 1 are updated. In addition, when the history image previously written remains in the frame memory, the history image is displayed in the blind spot area on the display screen 11.

The history image remaining on the frame memory is displayed in the blind spot area of the display screen 11 for a predetermined time after the vehicle 1 is stopped, but after the predetermined time has elapsed, the history image on the frame memory is displayed. May be deleted so that the history image is not displayed on the display screen 11.

As described above, when the image on the frame memory is displayed on the display screen 11 (S115), it is determined whether or not the display of the overhead image for the peripheral area of the vehicle 1 is terminated (S116). As a result, if the display is not finished yet (S116: NO), the process returns to the top of the process, and again after acquiring the captured images obtained by the in-

vehicle cameras

10F and 10R (S100 in FIG. 6), the above-described continues. A series of processing is executed. On the other hand, when it is determined that the display is to be ended (S116 in FIG. 7: YES), the image display process of the present embodiment is ended.

FIG. 8 illustrates a bird's-eye view image displayed on the display screen 11 by the image display process of the present embodiment described above. For convenience of explanation, also in FIG. 8, as in the case shown in FIG. 5, two white lines parallel to the traveling direction of the vehicle 1 and right angles to the two white lines are present on the ground in front of the vehicle 1. It is assumed that a single white line is drawn. Further, it is assumed that the moving body is moving in the same manner as in the case shown in FIG. In FIG. 5, the history image is displayed with a diagonal line, but in FIG. 8, the history image is displayed without a diagonal line in order to avoid complexity.

As apparent from a comparison between FIG. 8A and FIG. 5A, or FIG. 8B and FIG. 5B, while the moving body is in the imaging region of the in-vehicle camera 10F, this The bird's-eye view image displayed in the image display process of the embodiment is the same as the conventional bird's-eye view image.

As shown in FIG. 8C, when the moving body enters the blind spot area and is displayed as a history image, in this embodiment, the moving body is displayed in a mode different from the normal mode. In the illustrated example, the moving body is displayed in a color different from normal, such as red or yellow. For this reason, the driver thinks what happened and confirms the surroundings of the vehicle 1 by looking directly at the moving body, not through the display screen 11, so that the correct position of the moving body is recognized. Can do.

Similarly in FIG. 8D, the moving body is displayed on the display screen 11 in a mode different from the normal mode. As a result, the driver tries to visually confirm the moving body, and as a result, the correct position of the moving body is obtained. Recognize For this reason, even if the moving body is not displayed at the correct position on the display screen 11, it is possible to prevent the driver from misunderstanding the position of the moving body.

In the example shown in FIG. 8, it is assumed that when a moving object is present in the history image, only the moving object is displayed in a mode different from the normal mode. However, it is sufficient if the moving object in the history image can be displayed in a mode different from the normal mode. For example, when a moving body exists in the history image, the entire history image may be displayed in a mode different from the normal mode.

FIG. 9 shows a modification in which a history image is displayed in such a mode. In FIG. 9A and FIG. 9B, the moving body is present in the captured bird's-eye view image, and is not present in the left-right history image of the vehicle 1. For this reason, any history image in the left-right direction is displayed in the same mode as in the normal mode, similar to the captured bird's-eye view image.

However, in FIGS. 9C and 9D, there is a moving body in the history image on the right side of the vehicle 1. Therefore, the history image on the right side of the vehicle 1 is displayed in a mode different from the normal mode, for example, a reddish color or a yellowish color. Moreover, since there is no moving object, the left side history image of the vehicle 1 is displayed in the same mode as usual.

By doing this, the history image on the right side of the vehicle 1 is displayed on the display screen 11 in a mode different from the normal mode, so it is thought that what has happened, and the region on the right side of the vehicle 1 is directly checked by visual observation. . As a result, it is possible to recognize the correct position of the moving body.

In FIG. 9, the history image on the side where the moving body is present with respect to the vehicle 1 is displayed in a mode different from the normal mode. Therefore, for example, as shown in FIGS. 9B to 9D, when there is a portion where no history image is displayed in the blind spot area on the side of the vehicle 1, only the portion of the history image is normally displayed. Is displayed in a different mode.

However, the entire blind spot area on the side where the moving object exists may be displayed in a mode different from the normal mode. Even in this case, the history image on the side where the moving body exists is displayed in a mode different from the normal mode, and the mobile body itself is displayed in a mode different from the normal mode. As a result, since the driver tries to confirm the moving body by visual observation, the correct position of the moving body can be recognized.

Although the present disclosure has been described based on the embodiment, it is understood that the present disclosure is not limited to the embodiment or the structure. The present disclosure includes various modifications and modifications within the equivalent range. In addition, various combinations and forms, as well as other combinations and forms including only one element, more or less, are within the scope and spirit of the present disclosure.

Claims

A vehicle peripheral image display device for displaying an image obtained by photographing a peripheral region of a vehicle (1) using an in-vehicle camera (10F, 10R) on a display screen (11),
A photographic bird's-eye view image that generates a photographic bird's-eye view image by performing line-of-sight conversion for converting the peripheral region captured by the vehicle-mounted camera into an image photographed from above the vehicle with respect to the photographed image obtained by the vehicle-mounted camera. A generator (102);
A moving body detection unit (103) that detects a moving body from the photographed bird's-eye view image or the photographed image, and stores the photographed bird's-eye view image, and when the moving object is present in the photographed bird's-eye view image, the photographed bird's-eye view image. A photographing overhead image storage unit (104) that also stores the position of the moving body in
A shooting bird's-eye view image display unit (106) for displaying the shooting bird's-eye view image at a position corresponding to the peripheral area shot by the vehicle-mounted camera on the display screen;
A movement information acquisition unit (105) for acquiring movement information relating to the amount of movement of the vehicle;
A blind spot area that is a blind spot of the in-vehicle camera in the peripheral area of the vehicle is generated from the photographed bird's-eye view image generated before the vehicle reaches the current position and stored in the photographed bird's-eye view image storage unit. A history image that is an image of a part where the blind spot area is imaged is cut out based on the movement information and displayed at a position corresponding to the blind spot area on the display screen;
When the moving object is present in the history image, the history image display unit displays the moving object in a display mode different from the display mode displayed by the photographing overhead image display unit. Display device.
The vehicle periphery image display device according to claim 1,
The vehicle-mounted camera is mounted on at least one of the front or rear of the vehicle.
The vehicle periphery image display device according to claim 2,
The history image display unit
The history image is displayed for each of the blind spot area present on the right side of the vehicle and the blind spot area present on the left side of the vehicle,
When the moving body exists in the history image, the history image on the side where the moving body exists is displayed in a display mode different from the history image on the side where the moving body does not exist. Display device.
A vehicle peripheral image display method for displaying an image obtained by photographing a peripheral region of a vehicle using an in-vehicle camera on a display screen,
A photographic bird's-eye view image that generates a photographic bird's-eye view image by performing line-of-sight conversion for converting the peripheral region captured by the vehicle-mounted camera into an image photographed from above the vehicle with respect to the photographed image obtained by the vehicle-mounted camera. Generating step (S101);
A moving body detection step (S102) for detecting a moving body from the photographed bird's-eye view image or the photographed image; storing the photographed bird's-eye view image; and if the moving object is present in the photographed bird's-eye view image, the photographed bird's-eye view image. A photographing overhead image storage step (S104, S105) for storing the position of the moving body in
On the display screen, a shooting overhead image display step (S106) for displaying the shooting overhead image at a position corresponding to the peripheral area captured by the in-vehicle camera;
A movement information acquisition step (S107) for acquiring movement information relating to the amount of movement of the vehicle;
A blind spot area that is a blind spot of the in-vehicle camera in the peripheral area of the vehicle is generated from the photographed bird's-eye view image generated before the vehicle reaches the current position and stored in the photographed bird's-eye view image storing step. A history image display step (S110, S111, S113, S110, S111, S113, and a history image that is an image of a portion of the blind spot area that is cut out based on the movement information and displayed at a position corresponding to the blind spot area on the display screen. S114) and
In the history image display step, when the moving body is present in the history image, the second display mode is different from the first display mode displayed by the photographing overhead image display unit for the moving body. A vehicle periphery image display method, which is a step of displaying at.