JP6464846B2 - Vehicle surrounding image display control device and vehicle surrounding image display control program - Google Patents

Description

  The present invention relates to a vehicle surrounding image display control device that displays a vehicle surrounding image on a display unit, and a vehicle surrounding image display control program that causes the control device to function as a vehicle surrounding image display control device.

  2. Description of the Related Art Conventionally, a technique is known in which a camera attached to a vehicle is repeatedly photographed around the vehicle, the photographed image is converted into a bird's-eye view image viewed from above the vehicle, and the bird's-eye image is displayed on a display unit.

In such a technique, a technique for expanding the image display range around the vehicle by displaying a synthesized bird's-eye image obtained by combining the bird's-eye image obtained from the latest photographed image and the bird's-eye image obtained from the past photographed image on the display unit. Is known (for example, Patent Documents 1 and 2)
In Patent Document 1, a bird's-eye view image obtained from a captured image is stored in a history area of a memory. And a steering angle, a vehicle speed, etc. are acquired, the motion of a vehicle is calculated, and the bird's-eye view image memorize | stored in the log | history area | region is moved so that it may move contrary to the calculated motion of the vehicle. Then, a synthesized bird's-eye image is generated by synthesizing the bird's-eye image obtained from the latest photographed image and the bird's-eye image stored in the history area, and this synthesized bird's-eye image is displayed on the display unit.

  The process of moving the bird's-eye view image stored in the history area corresponding to the movement of the vehicle and superimposing the bird's-eye view image obtained from the latest captured image with the bird's-eye view image stored in the history area is repeated. Thereby, as the vehicle moves, the bird's-eye view image stored in the history area surrounds the vehicle periphery, so that the portion where the synthesized bird's-eye image is present around the vehicle is enlarged.

JP 2013-246493 A JP 2002-373327 A

  Hereinafter, a memory having a history area is referred to as a history memory. The history memory may include an area for storing an image taken by the camera in addition to the history area.

  Patent Document 1 does not describe whether all bird's-eye images obtained from captured images are stored in a history memory or whether a part is stored in a history memory. If the entire range of the photographed image is bird's-eye converted, and all the bird's-eye images obtained by the conversion are stored in the history memory, the storage process takes time because the image size to be stored is large. The required memory size also increases. Also, the image quality is poor in the peripheral part of the bird's-eye view image.

  Furthermore, there is a possibility of memorizing the part where the bumper or trailer hook is shown. If you remember the part where the bumper or trailer hook is shown, if you move the bird's-eye view image stored in the history memory according to the movement of the vehicle, it will be in a location away from the vehicle. There is a hook.

  If the image to be stored is reduced, the time required for the storage process can be shortened. However, if the image to be stored is reduced, a gap is generated between the image stored in the history memory at a certain time T and the image stored in the history memory at the time T + 1. As a result, in the history image stored in the history memory, there is a possibility that a portion having no image is generated in an area used for generating the synthesized bird's-eye view image.

  The present invention has been made based on this situation, and the object of the present invention is to suppress the occurrence of a lack in the area used for generating the synthesized bird's-eye image in the history image stored in the history memory, and Another object of the present invention is to provide a vehicle surrounding image display control device and a vehicle surrounding image display control program that can shorten the processing time for storing images and reduce the memory size for storing images.

  The above object is achieved by a combination of the features described in the independent claims, and the subclaims define further advantageous embodiments of the invention. Reference numerals in parentheses described in the claims indicate a correspondence relationship with specific means described in the embodiments described later as one aspect, and do not limit the technical scope of the present invention. .

  The invention related to the vehicle surrounding image display control device for achieving the above object is the acquisition unit (S1) for sequentially acquiring captured images obtained by photographing the surroundings of the vehicle from the cameras (10, 20) mounted on the vehicle (2). And the history in the area where the vehicle is not captured from the bird's-eye conversion unit (51) that sequentially converts the bird's-eye view of the captured image acquired by the acquisition unit to generate the bird's-eye view image, and the bird's-eye image sequentially generated by the bird's-eye conversion unit. A storage processing unit (52) that sequentially cuts out images and stores them in the history memory (43), a history moving unit (56) that moves the history images stored in the history memory in accordance with the movement of the vehicle, and a bird's eye conversion unit From the current image area (34, 35) obtained by cutting out the current image area from the latest bird's-eye view image converted by and the history image stored in the history memory, the history area (81 ) An image combining unit that combines the cut-out history image portion (33) to generate a combined bird's-eye image (31) that is a bird's-eye view image including an area that is not captured in the latest bird's-eye view image around the vehicle. (57) and an image output unit (58) that outputs the synthesized bird's-eye view image generated by the image synthesis unit to the display unit (30), and a history that the storage processing unit cuts out The image is an image having a size equal to or larger than the minimum rectangle and an area determined based on the minimum rectangle. The minimum rectangle is a rectangle having a pair of sides parallel to the vehicle width direction of the vehicle. One end of the side (63a) on the vehicle side of the pair of sides parallel to the vehicle width direction is on the inner circumference of the movement locus of the history area that moves when the vehicle turns to the right with the minimum turning radius, The other end is when the vehicle turns left with the minimum turning radius. It is located on the inner circle of the moving locus of the moving history area, and the length of the smallest rectangular vehicle in the front-rear direction is the last time the history image was captured since the last time the captured image corresponding to the bird's-eye view image was cut out. The movement length of the history area is caused by the movement of the vehicle until the captured image corresponding to the bird's-eye image to be cut out is captured.

  Moreover, the invention which concerns on the vehicle surroundings image display control program for achieving the said objective acquires a picked-up image which image | photographed the circumference | surroundings of a vehicle sequentially from the camera (10, 20) mounted in the vehicle (2). The vehicle is shown from the acquisition unit (S1), the bird's-eye conversion unit (51) that sequentially converts the captured image acquired by the acquisition unit to bird's-eye view and generates the bird's-eye view image, and the bird's-eye image that is sequentially generated by the bird's-eye conversion unit. A history processing unit (52) that sequentially cuts out history images in an unoccupied area and stores them in the history memory (43), and a history moving unit (56) that moves the history images stored in the history memory in accordance with the movement of the vehicle. And the current image area (34, 35) obtained by cutting out the current image area from the latest bird's-eye image converted by the bird's-eye conversion unit, and the history image stored in the history memory, the vehicle side more than the current image area. The history image portion (33) obtained by cutting out the history area (81) that is a) is synthesized, and a synthesized bird's-eye image that is an aerial image including an area that is not captured in the latest bird's-eye image around the vehicle. A vehicle surrounding image display control program that functions as an image synthesis unit (57) that generates (31) and an image output unit (58) that outputs a synthesized bird's-eye view image generated by the image synthesis unit to the display unit (30). The history image cut out by the storage processing unit is an image whose size is equal to or larger than the minimum rectangle and the area is determined based on the minimum rectangle. The minimum rectangle is a rectangle whose pair of sides are parallel to the vehicle width direction of the vehicle. Of the pair of sides parallel to the vehicle width direction of the smallest rectangle, the side (63a) on the vehicle side is one of the movement trajectories of the history area where one end moves when the vehicle turns right with the minimum turning radius. On the inner circle, the other end The vehicle's front-rear direction length is on the inner circle of the movement trajectory of the history area that moves when the vehicle turns to the left with the minimum turning radius. The movement length of the history area, which is caused by the movement of the vehicle from the time when the vehicle is photographed until the time when the photographed image corresponding to the bird's-eye view image from which the history image is cut out, is photographed.

  According to these inventions, the history image cut out by the storage processing unit is an image obtained by cutting out an area where no vehicle is shown from the bird's-eye view image. Therefore, even if the history moving unit moves the history image according to the movement of the vehicle, the bumper and the trailer hook do not exist at a place away from the vehicle.

  The history image cut out by the storage processing unit is an image whose area is determined based on the minimum rectangle. Of the pair of sides parallel to the vehicle width direction of the smallest rectangle, the side on the vehicle side is on the inner circumference of the movement locus of the history area that moves when the vehicle turns right with the minimum turning radius, The other end is on the inner circumference of the movement trajectory of the history area that moves when the vehicle turns left with the minimum turning radius.

  Therefore, if a minimum rectangular area or larger area is cut out as a history image from the bird's-eye view image, even if the vehicle moves most to the left and right and the history area moves with the movement, the history image stored in the history memory is In the history area, the occurrence of a portion having no history image in the vehicle width direction is suppressed.

  In addition, the length of the vehicle in the front-rear direction of the minimum rectangle is determined from the time when the captured image corresponding to the bird's-eye image from which the history image is cut is captured until the time when the captured image corresponding to the bird's-eye image from which the history image is cut is captured. This is the movement length of the history area caused by the movement of the vehicle.

  Therefore, if a minimum rectangular area or larger area is cut out as a history image from the bird's-eye view image, the history image stored in the history memory is the history image in the vehicle front-rear direction in the history area even if the vehicle moves in the front-rear direction. It is suppressed that the part which does not have is produced. As described above, the history image stored in the history memory is restrained from generating a portion having no image in the history area.

  In addition, the history image cut out by the storage processing unit is an image whose area is determined based on the minimum rectangle, and thus is a smaller image than the bird's-eye image obtained by bird's-eye conversion of the entire range of the photographed image. Therefore, the memory size required for the history memory can be reduced, and the processing time for storing the history image can be shortened.

1 is a diagram illustrating an overall configuration of a vehicle surrounding image display system 1. FIG. 4 is a diagram illustrating a synthesized bird's-eye image 31 displayed on the display unit 30. FIG. It is a flowchart explaining the process which the control apparatus 50 performs. It is a flowchart which shows the process of FIG.3 S3 in detail. It is a figure explaining the back picked-up image 70 which the rear camera 20 image | photographed. It is a figure explaining the back current image 72 which carried out the bird's-eye conversion of the back picked-up image. It is a figure explaining the position and magnitude | size of the cutting-out area 63 cut out as a log | history image.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(overall structure)
A vehicle surrounding image display system 1 shown in FIG. 1 is mounted on a vehicle 2, and includes a front camera 10, a rear camera 20, a display unit 30, an image memory 41, a recognition memory 42, a history memory 43, and a control device 50. ing.

  The front camera 10 is fixed to the front end of the vehicle 2 and continuously captures a predetermined range from the vicinity of the front end of the vehicle 2 to the front of the vehicle 2 at a predetermined cycle. Hereinafter, an image captured by the front camera 10 is referred to as a forward captured image.

  The rear camera 20 is fixed to the rear end of the vehicle 2 and continuously captures a predetermined range from the vicinity of the rear end of the vehicle 2 to the rear of the vehicle 2 at a predetermined cycle. Hereinafter, an image captured by the rear camera 20 is referred to as a rear captured image 70 (see FIG. 5). For the front camera 10 and the rear camera 20, for example, cameras equipped with fisheye lenses are used. Which range the front camera 10 and the rear camera 20 capture with respect to the vehicle 2 is stored in the control device 50 in advance.

  A synthetic bird's-eye view image 31 illustrated in FIG. 2 and a vehicle graphic 32 representing the vehicle 2 are displayed on the display unit 30. The vehicle graphic 32 is a graphic representing the vehicle 2 and is stored in advance in a predetermined memory. As can be seen from the vehicle figure 32 being displayed at the center of the synthesized bird's-eye image 31, the synthesized bird's-eye image 31 is an image around the vehicle.

  The image memory 41, the recognition memory 42, and the history memory 43 are all rewritable memories. Note that these may be three physically different memories, or an address of one memory may be divided into an image memory 41, a recognition memory 42, and a history memory 43.

  The image memory 41 stores a bird's-eye image obtained by bird's-eye conversion of a part of a front-shot image taken by the front camera 10 and a bird's-eye image obtained by bird-eye-transforming a part of a rear-shot image 70 taken by the rear camera 20. Is done. Hereinafter, the bird's-eye view image stored in the image memory 41 is referred to as a current image in order to distinguish it from the bird's-eye view image stored in the history memory 43. Also, among the current images, an image obtained by converting the front captured image is referred to as a front current image, and an image obtained by converting the rear captured image 70 is referred to as a rear current image 72 (see FIG. 6).

  The recognition memory 42 stores a recognition image that is an image obtained by cutting out a preset range from the bird's-eye view image stored in the image memory 41. A history image is stored in the history memory 43. The history image is an image obtained by combining a part of the recognition image cut out from the recognition image and the history image already stored in the history memory 43. In the history image already stored in the history memory 43, relative coordinates around the vehicle 2 are sequentially moved as the vehicle 2 moves.

  The control device 50 corresponds to a vehicle surrounding image display control device. The control device 50 is a computer including a CPU, a ROM, a RAM, and the like. The CPU executes a vehicle surrounding image display control program stored in a non-transitional tangible recording medium such as a ROM while using the temporary storage function of the RAM. Thereby, the control device 50 includes a bird's-eye conversion unit 51, a storage processing unit 52, a movement distance estimation unit 53, a rudder angle estimation unit 54, a vehicle movement amount calculation unit 55, a history movement unit 56, an image composition unit 57, and an image output unit. 58 functions. When these functions are executed, a method corresponding to the program is executed.

  These functions will be described with reference to the flowchart shown in FIG. Note that some or all of the functions executed by the control device 50 may be configured in hardware by one or a plurality of ICs.

(Function of control device 50)
FIG. 3 is a flowchart illustrating processing executed by the control device 50. The control device 50 periodically executes the process shown in FIG. 3 when an instruction signal instructing to display the synthesized bird's-eye image 31 on the display unit 30 is input. The instruction signal is generated, for example, by a user's switch operation. It also occurs when the shift position becomes the rear position.

  Steps S <b> 1 and S <b> 2 are processes performed by the bird's eye conversion unit 51. Step S <b> 1 corresponds to an acquisition unit in claims, and acquires a front shot image from the front camera 10. Further, the rear shot image 70 is acquired from the rear camera 20.

  In step S2, the front shot image and the rear shot image 70 acquired in step S1 are respectively bird's-eye converted, and the front current image and the rear current image 72 obtained by bird's-eye conversion are stored in the image memory 41. If the past forward current image and the backward current image 72 are already stored in the image memory 41, the past forward current image and the backward current image 72 are deleted, and only the latest forward current image and the backward current image 72 are deleted. Is stored in the image memory 41.

  By the bird's-eye view conversion, the front shot image and the rear shot image 70 are converted into an image looked down from the viewpoint above the vehicle 2. The conversion formula used for bird's-eye conversion is stored in a memory such as a ROM of the control device 50. This conversion formula is a well-known formula described in Patent Document 2, for example.

  Of the front captured image and the rear captured image 70, the range for bird's eye conversion is a predetermined range so as to include a recognition image cutout area 62 described later, and is a partial range of the front captured image and the rear captured image 70. is there. In the recognized image cutout area 62, the range to be cut out in the front shot image and the rear shot image 70 is determined in advance, so that the range for bird's eye conversion can also be set in advance.

  FIG. 5 is a diagram illustrating the rear shot image 70, and FIG. 6 conceptually shows the rear current image 72 obtained by bird's-eye conversion of the image of FIG. 5 in step S2. Since the rear shot image 70 is an image that is larger as it is closer to the vehicle 2, the pair of parking frame lines 61 are more distant from each other on the image as the vehicle 2 side becomes the lower side in FIG. 5. In contrast, as shown in FIG. 6, in the rear current image 72 that is a bird's-eye view image, the pair of parking frame lines 61 are parallel to each other.

  In step S3, a history movement calculation is performed. This history movement calculation is an operation for moving the relative coordinates of the history image stored in the history memory 43. The history movement calculation is shown in detail in FIG.

  In FIG. 4, a vehicle signal is acquired in step S31. Specifically, the vehicle signal is a vehicle speed signal or a steering angle signal. These signals are acquired via an in-vehicle LAN or the like. The vehicle speed signal is acquired from the vehicle speed sensor, and the steering angle signal is acquired from the steering angle sensor. Note that a yaw rate signal may be acquired instead of the steering angle signal.

  Step S32 is a process performed by the movement distance estimation unit 53. In this step S32, from the vehicle speed represented by the vehicle speed signal acquired in step S31 and the elapsed time since the previous execution of step S32, from the previous execution of step S32 to the current execution of step S32. The distance traveled by the vehicle 2 is calculated.

  Step S33 is a process performed by the rudder angle estimation unit 54. In step S33, the current steering angle of the vehicle 2 is estimated based on the steering angle signal acquired in step S31.

  Steps S34 and S35 are processes performed by the vehicle movement amount calculation unit 55. In step S34, a shift position signal is acquired from the shift position sensor. In step S35, the vehicle 2 moves from the previous execution of step S35 to the current execution of step S35, based on the movement distance and steering angle estimated in steps S32 and S33, and the shift position signal acquired in step S34. The amount of vehicle movement performed is calculated. This amount of movement includes the direction of movement relative to the previous position of the vehicle 2 and is a value that determines relative coordinates with respect to the previous position of the vehicle 2.

  Step S36 is processing performed by the history moving unit 56. In this step S36, the relative coordinates of the history image stored in the history memory 43 are moved in the direction opposite to the moving direction of the vehicle 2 by the vehicle movement amount calculated in step S35.

  Returning to FIG. Step S <b> 4 is processing performed by the storage processing unit 52. The recognition image is cut out from the current image stored in the image memory 41 and stored in the recognition memory 42. As described above, since the front current image and the rear current image 72 are stored as the current image, a recognition image is cut out from each current image.

  The recognition image has a size including a history image, and an area (hereinafter, recognized image cutout area) 62 cut out from the current image is set in advance. FIG. 6 illustrates a recognized image cutout area 62. The recognition processing unit 3 uses the recognition image.

  The recognition processing unit 3 performs image recognition processing such as white line recognition processing on the recognition image. By this white line recognition processing, for example, the parking frame line 61 is recognized. The recognized parking frame line 61 is further used for automatic parking control and the like.

  Step S5 is also processing performed by the storage processing unit 52. In step S 5, a history image is cut out from the recognition image stored in the recognition memory 42 and stored in the history memory 43. The recognition image is cut out from the front current image and the rear current image 72 and stored in the recognition memory 42. In step S5, a history image is cut out from each of the recognition image cut out from the front current image and the recognition image cut out from the rear current image 72.

  The processing in FIG. 3 is repeatedly executed. In the second and subsequent executions of FIG. 3, the history image acquired in the past is already stored in the history memory 43, and the history image acquired in the past is stored in step S3. In the history movement calculation, relative coordinates are moved by the amount of movement of the vehicle 2. The history image newly stored in the history memory 43 in step S5 is superimposed on the history image already stored in the history memory 43. A history image obtained by superimposing these is hereinafter referred to as a combined history image. When creating a composite history image, if the newly stored history image and the already stored history image are at the same relative coordinates, the newly stored history image is overwritten.

  FIG. 6 shows a history image cutout area (hereinafter simply referred to as a cutout area) 63 cut out as a history image. FIG. 6 also shows a vehicle range 64. The vehicle range 64 is a figure for explaining the range of the vehicle 2 with respect to the rear current image 72.

  In FIG. 6, as can be seen from the fact that the cutout area 63 and the vehicle range 64 do not overlap, the cutout area 63 is an area where a part of the vehicle 2 such as a bumper or a trailer hook is not shown.

  The cut-out area 63 is rectangular, and the long side is parallel to the width direction of the vehicle 2 and the short side is parallel to the front-rear direction of the vehicle 2. The cut-out area 63 is sized so that the combined history image is not missing when the relative coordinates of the history image sequentially move as the vehicle 2 travels and a new history image is stored in the history memory 43. Is set. Detailed description of the cut-out area 63 will be described later.

  Steps S6 to S8 are processing performed by the image composition unit 57. In step S6, the vehicle plane size is acquired. The vehicle plane size includes a vehicle length and a vehicle width, and is stored in advance in a predetermined memory. In step S7, the history image portion 33 (see FIG. 2) used by the output image is acquired from the combined history image stored in the history memory 43. The history image portion 33 is a portion sandwiched between the front current image portion 34 and the rear current image portion 35 in the synthesized bird's-eye image 31 displayed on the display unit 30. Since the history image portion 33 is a range not captured in the current image, the history image is used.

  In step S8, the front current image area is cut out from the front current image stored in the image memory 41 to form the front current image portion 34 shown in FIG. 2, and the rear current image area is cut out in the image memory 41 and shown in FIG. The rear current image portion 35 is assumed. Ranges are set in advance for the front current image area and the rear current image area. In step S8, the front current image portion 34 and the rear current image portion 35 are combined with the history image portion 33 acquired in step S7 to generate a composite bird's-eye view image 31. Step S9 is a process performed by the image output unit 58, and the composite bird's-eye image 31 generated in step S8 is output to the display unit 30 as an output image.

(Description of cut-out area 63)
As described above, the cutout area 63 is an area that is cut out to generate a combined history image, and the combined history image is an image for cutting out the history image portion 33. The portion that becomes the history image portion 33 in the combined history image moves as the vehicle 2 moves. Therefore, the position and size of the cutout area 63 are set so that the history image portion 33 is not missing in the combined history image regardless of the movement of the vehicle 2.

  FIG. 7 is a diagram for explaining the position and size of the cut-out area 63. In FIG. 7, the turning axis line 80 passes through the axle of the rear wheel of the vehicle 2 and passes through the turning center when the vehicle 2 turns with the minimum turning radius. The history area 81 is an area cut out as a history image portion 33 from the combined history image. The history area 81 is a rectangle whose position and size are determined based on the vehicle 2.

  The right inner circular arc 82 is the inner end of the locus when the history area 81 is moved by turning the vehicle 2 to the right with the minimum turning radius, and the left inner circular arc 83 is the left turning with the minimum turning radius. This is the inner end of the locus when the history area 81 is moved by turning.

  In the rectangular cut-out area 63, the distance L between the vehicle side 63 a that is the vehicle side of the pair of sides parallel to the vehicle width direction and the front end of the vehicle 2 is determined by the imaging range of the rear camera 20. If the vehicle side 63a does not intersect the right inner circular arc 82 and the left inner circular arc 83, when the history area 81 moves as the vehicle 2 moves with the minimum turning radius, the history area 81 There is a possibility that a portion having no composite history image may be generated at the end of. In other words, if the vehicle side 63a intersects the right inner circular arc 82 and the left inner circular arc 83, even if the vehicle 2 moves with the minimum turning radius and the history area 81 moves accordingly, the history area 81 There is no portion where there is no composite history image in the left-right direction.

  Therefore, the vehicle side 63a of the cut-out area 63 has a distance L from the front end of the vehicle 2, is parallel to the width direction of the vehicle 2, has one end on the right inner circumferential arc 82, and the other end. It is set so as to be on the left inner circumferential arc 83.

  In the cutout area 63, the length of the vehicle longitudinal direction side 63b parallel to the vehicle longitudinal direction is determined based on the vehicle speed (m / s) and the history image storage cycle (s). That is, the length obtained by the product of the vehicle speed and the storage cycle is set as the length of the vehicle longitudinal direction side 63b.

  The history image is stored in the history memory 43 every time the front camera 10 and the rear camera 20 capture the front captured image and the rear captured image 70. Therefore, the vehicle speed × the storage cycle is the vehicle 2 from the time when the photographed image corresponding to the bird's-eye image obtained by cutting out the history image is taken until the time when the photographed image corresponding to the bird's-eye image obtained by cutting out the history image is taken this time. Means the movement length of the history area 81 caused by the movement.

  The rectangle determined in this way corresponds to the minimum rectangle of the claims. The cutout area 63 may be a range surrounded by the minimum rectangle, but may be an area that includes the minimum rectangle and is larger than the minimum rectangle in consideration of errors and the like.

  The description of FIG. 7 is a description of the cutout area 63 for cutting out the history image from the rear current image 72, but the cutout area for cutting out the history image from the front current image is set in the same way.

(Effect of embodiment)
In the present embodiment described above, the history image cut out by the storage processing unit 52 (S4, S5) is an image obtained by cutting out an area where the vehicle 2 is not shown from the bird's-eye view image. Therefore, even if the history moving unit 56 (S36) moves the history image in accordance with the movement of the vehicle 2, there is no bumper or trailer hook at a location away from the vehicle 2.

  The history image that the storage processing unit 52 (S4, S5) cuts out from the image stored in the recognition memory 42 and stores it in the history memory 43 is an image obtained by cutting out the cut-out area 63 determined based on the minimum rectangle.

  As described with reference to FIG. 7, when the cut-out area 63 is the same as the minimum rectangle, one end of the vehicle side 63a of the cut-out area 63 moves when the vehicle 2 turns right with the minimum turning radius. It is on the inner circle of the movement locus of the history area 81. The other end of the vehicle side 63a is on the inner circumference of the movement locus of the history area 81 that moves when the vehicle 2 turns left with the minimum turning radius. The cut-out area 63 is an area that is the same as or includes the minimum rectangle, and is larger than the minimum rectangle.

  Therefore, by cutting out the cut-out area 63 as a history image from the bird's-eye view image, even if the vehicle 2 moves most to the left and right and the history area 81 moves with the movement, the combined history image stored in the history memory 43 is In the history area 81, it is possible to suppress a portion having no image in the vehicle width direction.

  When the cutout area 63 is the same as the minimum rectangle, the length of the vehicle front-rear direction side 63b of the cutout area 63 is a length obtained by the product of the vehicle speed and the storage period of the history image. Therefore, as described above, the cut-out area 63 is an area that is the same as the minimum rectangle or includes the minimum rectangle and is larger than the minimum rectangle.

  Therefore, if the cut-out area 63 is cut out as a history image from the bird's-eye view image, even if the vehicle 2 moves in the front-rear direction, the composite history image stored in the history memory 43 has no image in the vehicle front-rear direction in the history area 81. Generation of a part is suppressed.

  As described above, the combined history image stored in the history memory 43 is prevented from having a portion having no image in the vehicle left-right direction and the vehicle front-rear direction in the history area 81.

  In addition, the history image cut out from the recognition memory 42 by the storage processing unit 52 is an image in which an area is determined based on the minimum rectangle, and thus is a smaller image than a bird's-eye image obtained by bird's-eye conversion of the entire range of the captured image. Therefore, the memory size of the history memory 43 can be reduced, and the processing time for storing the history image can be shortened.

  Further, the storage processing unit 52 first cuts out a recognition image from the current image stored in the image memory 41 and stores it in the recognition memory 42 (S4), and the recognition image stored in the recognition memory 42. A history image is cut out and stored in the history memory 43 (S5). By doing in this way, the image cut out from the present image can be used for recognition and for the synthesized bird's-eye image display.

  Further, the range in which the bird's-eye conversion unit 51 (S2) performs bird's-eye conversion is determined so as to include the recognized image cutout area 62, and is a partial range of the front shot image and the rear shot image 70. Therefore, the processing speed can be improved and the storage area required for the image memory 41 can be reduced as compared with the bird's-eye view conversion of the entire range of the front captured image and the rear captured image 70.

  As mentioned above, although embodiment of this invention was described, this invention is not limited to the above-mentioned embodiment, The following modification is also contained in the technical scope of this invention, Furthermore, the summary other than the following is also included. Various modifications can be made without departing from the scope.

<Modification 1>
For example, in the above-described embodiment, the recognition image is cut out from the current image stored in the image memory 41 and stored in the recognition memory 42, and the history image is cut out from the recognition memory 42. That is, an image cut out from the current image is stored in the history memory 43 via the recognition memory 42. However, a history image may be cut out from the current image and stored in the history memory 43.

<Modification 2>
In the above-described embodiment, the front camera 10 and the rear camera 20 are provided, but only one of the cameras may be provided.

<Modification 3>
In the above-described embodiment, the current image is stored in the image memory 41, and the recognition image is cut out from the image memory 41. However, the image memory 41 may not be provided, and the recognition image and the history image may be cut out from the image converted by the bird's eye conversion unit 51 and stored directly in the recognition memory 42 and the history memory 43.

<Modification 4>
The vehicle 2 according to the above-described embodiment includes the recognition memory 42 and the recognition processing unit 3. However, when it is not necessary to perform the image recognition processing, the recognition memory 42 and the recognition processing unit 3 are included in the vehicle 2. It does not have to be.

<Modification 5>
The image memory 41 or the recognition memory 42 may also be used as the history memory 43. That is, a part of the image memory 41 or the recognition memory 42 may be used as the history memory 43.

<Modification 6>
In the above-described embodiment, the history image is cut out from the recognition image stored in the recognition memory 42 and stored in the history memory 43. However, if the recognition image is smaller than the history image, contrary to the embodiment, first, the history image is stored in the history memory 43, and the recognition image is cut out from the history memory 43 and stored in the recognition memory 42. May be.

  By cutting out the recognition image from the history memory 43 in this way, the recognition image can be made smaller compared to the case where a history image larger than the recognition image is used as it is as the recognition image. it can.

  Note that, when a history image is cut out from the recognition memory 42 as in the embodiment, the history image can be made smaller than when a recognition image larger than the history image is used as it is as a history image. Processing time for generating an image can be shortened.

1: Vehicle surrounding image display system 2: Vehicle 3: Recognition processing unit 10: Front camera 20: Rear camera 30: Display unit 31: Composite bird's-eye view image 32: Vehicle figure 33: History image portion 34: Front current image portion 35: Back Current image portion 41: Image memory 42: Memory for recognition 43: History memory 50: Control device 51: Bird's eye view conversion unit 52: Storage processing unit 53: Travel distance estimation unit 54: Steering angle estimation unit 55: Vehicle movement amount calculation unit 56 : History moving unit 57: Image composition unit 58: Image output unit 61: Parking frame line 62: Recognition image cutout area 63: Cutout area 63a: Vehicle side 63b: Vehicle longitudinal direction 64: Vehicle range 70: Back shot image 72 : Back current image 80: Turning axis 81: History area 82: Right inner circular arc 83: Left inner circular arc

Claims (4)

An acquisition unit (S1) for sequentially acquiring captured images obtained by photographing the surroundings of the vehicle from cameras (10, 20) mounted on the vehicle (2);
A bird's eye conversion unit (51) for sequentially performing bird's eye conversion on the captured image acquired by the acquisition unit to generate a bird's eye image;
A storage processing unit (52) for sequentially cutting out a history image in an area in which the vehicle is not captured from the bird's eye image sequentially generated by the bird's eye conversion unit, and storing the history image in a history memory (43);
A history moving unit (56) for moving the history image stored in the history memory according to the movement of the vehicle;
From the current image area (34, 35) obtained by cutting out the current image area from the latest bird's-eye image converted by the bird's-eye conversion unit, and the history image stored in the history memory, the vehicle more than the current image area. The bird's-eye view image including an area that is not captured in the latest bird's-eye view image around the vehicle by combining the history image portion (33) obtained by cutting out the history area (81) that is the side area. An image composition unit (57) for generating a synthetic bird's-eye view image (31),
An image output unit (58) for outputting the synthesized bird's-eye image generated by the image synthesizing unit to a display unit (30);
The history image cut out by the storage processing unit is an image having a size equal to or larger than a minimum rectangle and an area determined based on the minimum rectangle,
The minimum rectangle is
A pair of sides is a rectangle parallel to the vehicle width direction of the vehicle,
The side (63a) on the vehicle side of the pair of sides parallel to the vehicle width direction of the minimum rectangle has one end of a movement locus of the history area that moves when the vehicle turns right with a minimum turning radius. The other end is on the inner circumference of the movement trajectory of the history area that moves when the vehicle turns to the left with a minimum turning radius,
The length of the vehicle in the front-rear direction of the minimum rectangle is the captured image corresponding to the bird's-eye image from which the history image is cut out this time after the captured image corresponding to the bird's-eye image from which the history image was cut out was previously captured. The vehicle surrounding image display control device, which is a movement length of the history area, which is caused by the movement of the vehicle before the image is taken. In claim 1,
The storage processing unit cuts out a recognition image including the history image from the bird's-eye image, stores it in a recognition memory (42) for storing the bird's-eye image for image recognition processing, and stores the recognition The vehicle surrounding image display control device, wherein the history image is cut out from the memory and stored in the history memory. In claim 2,
The bird's-eye view conversion unit performs bird's-eye conversion of an area determined based on the history image in the photographed image. Computer
An acquisition unit (S1) for sequentially acquiring captured images obtained by photographing the surroundings of the vehicle from cameras (10, 20) mounted on the vehicle (2);
A bird's eye conversion unit (51) for sequentially performing bird's eye conversion on the captured image acquired by the acquisition unit to generate a bird's eye image;
A storage processing unit (52) for sequentially cutting out a history image in an area in which the vehicle is not captured from the bird's eye image sequentially generated by the bird's eye conversion unit, and storing the history image in a history memory (43);
A history moving unit (56) for moving the history image stored in the history memory according to the movement of the vehicle;
From the current image area (34, 35) obtained by cutting out the current image area from the latest bird's-eye image converted by the bird's-eye conversion unit, and the history image stored in the history memory, the vehicle more than the current image area. The bird's-eye view image including an area that is not captured in the latest bird's-eye view image around the vehicle by combining the history image portion (33) obtained by cutting out the history area (81) that is the side area. An image composition unit (57) for generating a synthetic bird's-eye view image (31),
A vehicle surrounding image display control program for causing the composite bird's-eye image generated by the image composition unit to function as an image output unit (58) for outputting to a display unit (30),
The history image cut out by the storage processing unit is an image having a size equal to or larger than a minimum rectangle and an area determined based on the minimum rectangle,
The minimum rectangle is
A pair of sides is a rectangle parallel to the vehicle width direction of the vehicle,
The side (63a) on the vehicle side of the pair of sides parallel to the vehicle width direction of the minimum rectangle has one end of a movement locus of the history area that moves when the vehicle turns right with a minimum turning radius. The other end is on the inner circumference of the movement trajectory of the history area that moves when the vehicle turns to the left with a minimum turning radius,
The length of the vehicle in the front-rear direction of the minimum rectangle is the captured image corresponding to the bird's-eye image from which the history image is cut out this time after the captured image corresponding to the bird's-eye image from which the history image was cut out was previously captured. A vehicle surroundings image display control program characterized by the movement length of the history area, which is generated when the vehicle moves before the image is taken.

Images