JP2010233080A - Driving support device, driving support method, and driving support program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a driving support device, a driving support method and a driving support program that can output an image of a vicinity of a vehicle free of a discontinuous part and distortion while suppressing an increase in processing load without reference to an inclination of the vehicle. <P>SOLUTION: The driving support device 1 includes: a reference posture specifying portion 71 which specifies a reference posture to which a posture of the vehicle is referred; an image acquiring portion 72 that acquires image data of the vicinity of the vehicle; a photographic posture specifying portion 73 that specifies a photographic posture being a posture of the vehicle when the image acquiring portion 72 acquires the image data by photography; a deciding portion 74 that decides whether the image data that the image acquiring portion 72 acquires are usable on the basis of a difference between the reference posture that the reference posture specifying portion 71 specifies and the photographic posture that the photographic posture specifying portion 73 specifies; a bird's-eye view image creating portion 75 which creates a bird's-eye view image of an area including the vehicle using only image data that the deciding portion 74 decides to be usable; and a display 60 which displays the bird's-eye view image that the bird's-eye view image creating portion 75 creates. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a driving support device, a driving support method, and a driving support program.

  2. Description of the Related Art Conventionally, a driving support device that displays an image around a vehicle on a display is known as a device that supports driving of a driver. This driving support device displays image data around the vehicle photographed by a camera installed outside the vehicle on a display arranged near the driver's seat. There has also been proposed an apparatus for storing image data acquired from a camera in a memory and outputting a combined image obtained by combining the image data stored in the memory.

  In such a driving assistance device, when the vehicle is tilted due to the difference in the height of each tire of the vehicle and the in-vehicle camera is tilted from the reference posture, imaging is performed with the tilted posture. When image processing is performed using the processed image data, there is a problem that a discontinuous image or a distorted image is displayed. In response to this problem, there has also been proposed a driving support device that corrects image data captured by the camera in accordance with the tilt of the camera with respect to a reference posture and displays a driving support image using the corrected image data. (For example, refer to Patent Document 1).

JP 2007-266930 A (paragraph 0014)

  However, in the conventional driving support device as described above, it is necessary to perform correction in consideration of the tilt of the camera for all image data captured by the camera, and the load of the correction processing is increased. A high system was needed.

  The present invention has been made in view of the above, and is a driving support device capable of outputting an image of a vehicle periphery without discontinuities or distortion while suppressing an increase in processing load regardless of the inclination of the vehicle. An object is to provide a driving support method and a driving support program.

  In order to solve the above-described problems and achieve the object, the driving support apparatus according to claim 1 includes a reference posture specifying unit that specifies a reference posture that serves as a reference for the posture of the vehicle, and image data around the vehicle. The image acquisition unit for acquiring the image, the shooting posture specifying unit for specifying the shooting posture that is the posture of the vehicle when the image data acquired by the image acquisition unit is shot, and the reference posture specifying unit specify A determination unit that determines whether the image data acquired by the image acquisition unit is available based on a difference between the reference posture and the shooting posture specified by the shooting posture determination unit; An overhead image generation means for generating an overhead image of an area including the vehicle using only the image data determined to be usable; a display means for displaying the overhead image generated by the overhead image generation means; Provided.

  The driving support device according to claim 2 is the driving support device according to claim 1, wherein the image data acquired by the image acquisition unit and the shooting posture when the image data is shot. Are stored in association with each other, the reference attitude specifying means specifies a reference inclination as a reference of the inclination of the vehicle as the reference attitude, and the photographing attitude specifying means includes the The shooting inclination that is the inclination of the vehicle when the image data acquired by the image acquisition unit is captured is specified as the shooting posture, and the determination unit stores the shooting stored in the image data storage unit When the difference between the photographing inclination as the posture and the reference inclination is less than a threshold value, it is determined that image data corresponding to the photographing inclination is usable, and the overhead image generation unit is used by the determination unit Acquires image data determined as the ability of said image data storage means, to generate the bird's-eye view image using the acquired image data.

  According to a third aspect of the present invention, there is provided the driving support apparatus according to the second aspect, wherein the reference posture specifying means is the shooting specified by the shooting posture specifying means while the vehicle moves a predetermined distance. When the difference between the inclination and the reference inclination is continuously greater than or equal to the threshold value, the reference inclination is updated based on the photographing inclination.

  According to a fourth aspect of the present invention, there is provided the driving support apparatus according to the first aspect, further comprising a recording unit that records only the image data determined to be usable by the determination unit, and the overhead image generation unit includes: The overhead image is generated using the image data recorded in the recording means.

  According to a fifth aspect of the present invention, the driving support method includes a reference posture specifying step that specifies a reference posture that serves as a reference for the posture of the vehicle, an image acquisition step that acquires image data around the vehicle, and the image acquisition step. In the shooting posture specifying step for specifying the shooting posture, which is the posture of the vehicle when the image data acquired in (2) is shot, in the reference posture specified in the reference posture specifying step, and in the shooting posture specifying step A determination step for determining whether or not the image data acquired in the image acquisition step can be used based on the difference from the identified shooting posture, and only using the image data determined to be usable in the determination step. An overhead image generation step for generating an overhead image of an area including a vehicle, and a display screen for displaying the overhead image generated in the overhead image generation step. Tsu including and up, the.

  A driving support program according to a sixth aspect causes a computer to execute the method according to the fifth aspect.

  According to the driving support apparatus according to claim 1, the driving support method according to claim 5, and the driving support program according to claim 6, the image acquisition unit acquires based on the difference between the reference posture and the photographing posture. It is determined whether the image data can be used, and the overhead image of the area including the vehicle is generated using only the image data determined to be usable by the determination means. A bird's-eye view image can be generated using only image data having no image. Therefore, regardless of the vehicle inclination, it is possible to output an image of the vehicle periphery without discontinuities or distortion while suppressing an increase in processing load.

  According to the driving support apparatus of the second aspect, the image data storing unit associates the image data acquired by the image acquiring unit with the shooting posture when the image data is shot. And when the difference between the photographing inclination as the photographing posture stored in the image data storage means and the reference inclination is less than the threshold value, it is determined that the image data corresponding to the photographing inclination is usable. An overhead image can be generated by selecting only usable image data without distortion from image data that has been captured and stored in the image data storage means.

  Further, according to the driving support device according to claim 3, when the difference between the shooting inclination specified by the shooting posture specifying means and the reference inclination is continuously equal to or greater than the threshold while the vehicle moves a predetermined distance, Since the reference inclination is updated based on the photographing inclination, image data based on an appropriate reference inclination even when the road surface changes, for example, when the vehicle enters a slope road from a flat road. It can be determined whether or not is available.

  According to the driving support apparatus of the fourth aspect, the overhead image is generated using the image data recorded in the recording unit that records only the image data determined to be usable by the determination unit. The volume of image data to be recorded can be reduced.

1 is a block diagram illustrating a driving support device according to a first embodiment. It is a flowchart of an image display process. It is a flowchart of a reference | standard inclination update process. FIG. 4 is a diagram schematically showing the attitude of the vehicle and image data acquired by the image acquisition unit, and FIG. 4A is a side view of the vehicle when the difference between the reference inclination and the imaging inclination is less than a threshold value, FIG. 4 (b) is a diagram showing image data obtained by bird's-eye conversion of the image captured in the state of FIG. 4 (a), and FIG. 4 (c) is a side view of the vehicle when the difference between the reference inclination and the imaging inclination is a threshold value or more. FIG. 4 and FIG. 4D are diagrams showing image data obtained by bird's-eye conversion of an image taken in the state of FIG. FIG. 5A is a diagram illustrating a composite image, FIG. 5A is a diagram illustrating the composite image stored in the drawing memory, and FIG. 5B is a diagram illustrating the composite image displayed on the display 60. FIGS. 6A and 6D are diagrams conceptually showing a configuration of a composite image generated by image generation processing, and FIGS. 6A and 6D are diagrams showing a relationship between a range of an overhead image generated from image data and shooting. 6 (b) and 6 (e) are diagrams showing the relationship between the range used for composition and shooting at the time of each overhead view image, and FIGS. 6 (c) and 6 (f) are diagrams showing the generated composite image. is there. It is a flowchart of an image generation process.

  Hereinafter, embodiments of a driving support apparatus according to the present invention will be described in detail with reference to the drawings. However, the present invention is not limited to each embodiment.

[Embodiment 1]
First, the first embodiment will be described. In the first embodiment, the image data and the shooting posture when the image data is shot are stored in the image data storage unit in association with each other.

(Constitution)
First, the configuration of the driving support device will be described. FIG. 1 is a block diagram illustrating a driving assistance apparatus according to the first embodiment. As shown in FIG. 1, the driving support apparatus 1 includes a tilt sensor 10, a vehicle speed sensor 20, a steering sensor 30, a shift position sensor 40, a camera 50, a display 60, a control unit 70, and a data recording unit 80. .

(Configuration-Tilt sensor)
The tilt sensor 10 detects the tilt of the vehicle and outputs the detected value to the control unit 70. As this inclination detection means, for example, a known vehicle height sensor for detecting the vehicle height of each part of the vehicle from the road surface or a known acceleration sensor can be used.

(Configuration-vehicle speed sensor)
The vehicle speed sensor 20 outputs a vehicle speed pulse signal or the like proportional to the rotational speed of the axle to the control unit 70, and a known vehicle speed sensor can be used.

(Configuration-steering sensor)
The steering sensor 30 detects the steering angle of the steering wheel of the vehicle and outputs the detected value to the control unit 70, and a known steering sensor can be used.

(Configuration-Shift position sensor)
The shift position sensor 40 detects the position of the shift lever for operating the transmission of the vehicle and outputs the detected value to the control unit 70, and a known shift position sensor can be used.

(Configuration-Camera)
The camera 50 is a photographing unit that photographs the periphery of the vehicle. For example, one or a plurality of cameras 50 are installed in the vicinity of the front and rear bumpers of the vehicle, and photograph the periphery of the vehicle. Image data captured by the camera 50 is input to the control unit 70. The specific configuration of the camera 50 is arbitrary, and is configured using a known imaging device such as a CMOS image sensor or a CCD image sensor, and a known optical system component such as a fisheye lens or a prism.

(Configuration-Display)
The display 60 is a display unit that displays an overhead image generated by the control unit 70. The specific configuration of the display 60 is arbitrary, and a known flat panel display such as a liquid crystal display or an organic EL display can be used.

(Configuration-control unit)
The control unit 70 is a control unit that controls the driving support device 1. Specifically, the control unit 70 is a CPU, various programs that are interpreted and executed on the CPU (a basic control program such as an OS, and an OS that is started and specified And an internal memory such as a RAM for storing the program and various data. In particular, the driving support program according to the present embodiment is installed in the driving support device 1 via an arbitrary recording medium or network, thereby substantially configuring each unit of the control unit 70.

  The control unit 70 includes a reference posture specifying unit 71, an image acquisition unit 72, a photographing posture specifying unit 73, a determination unit 74, an overhead image generation unit 75, and an image memory 76 in terms of functional concept. The reference posture specifying unit 71 is a reference posture specifying unit that specifies a reference posture that serves as a reference for the posture of the vehicle. The image acquisition unit 72 is image acquisition means for acquiring image data around the vehicle. The shooting posture specifying unit 73 is a shooting posture specifying unit that specifies a shooting posture that is the posture of the vehicle when the image data acquired by the image acquisition unit 72 is shot. The determination unit 74 is a determination unit that determines whether the image data acquired by the image acquisition unit 72 is usable. The overhead image generation unit 75 is an overhead image generation unit that generates an overhead image of an area including a vehicle. The image memory 76 is an image data storage unit that stores the image data acquired by the image acquisition unit 72 and the shooting posture when the image data is shot in association with each other. The image memory 76 also records the overhead image and the synthesized image generated by the overhead image generation unit 75. Details of processing executed by each component of the control unit 70 will be described later.

(Configuration-Data recording part)
The data recording unit 80 is a recording unit that records a program and various data necessary for the operation of the driving support device 1, and is configured using, for example, a hard disk (not shown) as an external storage device. However, any other recording medium including a magnetic recording medium such as a magnetic disk or an optical recording medium such as a DVD or a Blu-ray disk can be used instead of or together with the hard disk.

(Processing-Image generation processing)
Next, an image generation process executed by the driving support device 1 configured as described above will be described. FIG. 2 is a flowchart of the image generation process (in the following description of each process, step is abbreviated as “S”). This image generation processing is started, for example, when an image generation processing start instruction is input via a known input unit, or when the shift position sensor 40 detects that the shift position is set to reverse. .

  As shown in FIG. 2, when the image generation process is activated, the image acquisition unit 72 acquires image data around the vehicle from the camera 50 (SA1). Subsequently, the reference attitude specifying unit 71 specifies a reference inclination that serves as a reference for the inclination of the vehicle as a reference attitude (SA2). Here, the reference posture specifying unit 71 specifies the inclination of the vehicle when the image data acquired first by the image acquisition unit 72 after the start of the image generation process in SA1 as the reference inclination. For example, at SA1, the image acquisition unit 72 acquires image data from the camera 50. At the same time, the reference posture specifying unit 71 specifies the vehicle inclination specified based on the input from the inclination sensor 10 as the reference inclination, and the RAM ( (Not shown). Subsequently, the image acquisition unit 72 stores the image data acquired in SA1 and the reference posture, which is the shooting posture when the image data is shot, in the image memory 76 in association with each other (SA3). ). For example, information specifying the reference inclination specified in SA2 is added to the image data acquired in SA1 as a header and stored in the image memory 76.

  Next, the image acquisition unit 72 determines the presence / absence of an image composition trigger (SA4). For example, when a vehicle speed pulse signal is input from the vehicle speed sensor 20, it is determined that an image composition trigger has occurred. Alternatively, when the vehicle is stopped, it is determined that there has been a trigger for image composition when a predetermined time has elapsed since the previous image composition.

  As a result, when it is determined that there is no trigger for image synthesis (SA4, No), the image acquisition unit 72 continues the determination of SA4. On the other hand, when it is determined that there has been a trigger for image synthesis (SA4, Yes), the image acquisition unit 72 acquires image data around the vehicle from the camera 50 (SA5).

  Subsequently, the shooting posture specifying unit 73 specifies the shooting inclination, which is the inclination of the vehicle when the image data acquired by the image acquisition unit 72 is shot, as the shooting posture (SA6). For example, at SA5, the image acquisition unit 72 acquires image data from the camera 50, and at the same time, the imaging posture specifying unit 73 refers to the input from the inclination sensor 10, and uses the vehicle inclination specified based on the input as the imaging inclination. Identify.

  Next, the image acquisition unit 72 stores the image data acquired in SA5 and the shooting posture when the image data was shot in the image memory 76 in association with each other (SA7). For example, information specifying the shooting inclination specified in SA6 is added to the image data acquired in SA5 as a header and stored in the image memory 76.

  Next, the reference posture specifying unit 71 executes reference inclination update processing (SA8). Here, the reference inclination update process will be described. FIG. 3 is a flowchart of the reference inclination update process. As illustrated in FIG. 3, when the reference inclination update process is activated, the reference posture specifying unit 71 determines whether the value of the distance counter stored in the RAM is equal to or greater than a preset distance threshold value, for example. (SB1). Here, the “distance counter” corresponds to the distance the vehicle has traveled since the reference inclination was previously set. As the distance threshold, for example, a value corresponding to the distance between the front and rear tires of the vehicle can be used.

  As a result, when the value of the distance counter is equal to or greater than the distance threshold (SB1, Yes), the reference posture specifying unit 71 determines the shooting inclination and the reference inclination specified by the shooting posture specifying unit 73 while the vehicle moves a predetermined distance. It is determined whether or not the difference between and is continuously greater than or equal to a threshold value (SB2). For example, with respect to each image data acquired while the vehicle moves a distance corresponding to the distance threshold to the current position and stored in the image memory 76, a shooting inclination and a reference inclination when the image data is captured It is determined whether or not the absolute value of the difference is greater than or equal to a threshold value. Here, as the threshold value, for example, a value that can be easily recognized by the user because distortion and discontinuous portions that occur in an image around the vehicle generated based on the image data captured by the camera 50 becomes significant. it can.

  As a result, when it is determined that the difference between the shooting inclination and the reference inclination is continuously greater than or equal to the threshold while the vehicle moves a predetermined distance (SB2, Yes), the reference inclination is updated based on the shooting inclination (SB3). . For example, the photographing inclination specified in SA6 in FIG. 2 is set as a new reference inclination. Thereafter, the reference posture specifying unit 71 resets the distance counter (SB4) and returns to the main routine.

  On the other hand, as a result of the determination of SB1, if the value of the distance counter is not greater than or equal to the distance threshold (SB1, No), or as a result of the determination of SB2, the difference between the photographing inclination and the reference inclination is not continuously greater than the threshold. In the case (SB2, No), the reference posture specifying unit 71 obtains a value corresponding to the distance traveled by the vehicle after the previous reference inclination update process by a known method and adds it to the distance counter (SB5). Return to the routine.

  Returning to FIG. 2, after executing the reference inclination update process in SA8, the determination unit 74 determines whether or not the reference inclination is updated in the SA8 reference inclination update process (SA9). As a result, when it is determined that the reference inclination is not updated (SA9, No), the determination unit 74 determines whether the image data acquired in SA5 is available (SA10). For example, when the difference between the photographing inclination specified in SA6 and the reference inclination is less than the threshold value, it is determined that the image data can be used. Here, as the threshold value, for example, the same value as the threshold value in the reference inclination update process described above can be used.

  FIG. 4 is a diagram schematically showing the attitude of the vehicle and the image data acquired by the image acquisition unit 72. FIG. 4A shows the vehicle when the difference between the reference inclination and the imaging inclination is less than a threshold value. 4B is a diagram showing image data obtained by performing a bird's-eye conversion of the image photographed in the state of FIG. 4A, and FIG. 4C is a diagram in which the difference between the reference inclination and the photographing inclination is equal to or greater than a threshold value. FIG. 4D is a diagram showing image data obtained by performing a bird's-eye conversion of the image photographed in the state of FIG. 4C. 4B and 4D show image data when a road surface on which two straight lines are drawn parallel to the traveling direction of the vehicle 2 is photographed.

  As shown in FIG. 4A, in the image data obtained by bird's-eye conversion of an image captured when the difference between the reference inclination and the imaging inclination is less than the threshold, two straight lines drawn parallel to the road surface are shown in FIG. As shown in b), they are displayed in parallel. Therefore, it is determined that the image data as shown in FIG. 4B can be used. On the other hand, as shown in FIG. 4C, for example, image data obtained by bird's-eye conversion of a photographed image when the difference between the reference inclination and the photographing inclination becomes equal to or greater than a threshold value due to the rear wheel of the vehicle 2 climbing a step. Then, the two straight lines drawn parallel to the road surface are distorted and displayed non-parallel as shown in FIG. Therefore, it is determined that the image data as shown in FIG. 4D is not usable.

  Returning to FIG. 2, when it is determined that the image data can be used as a result of the determination of SA10 (SA10, Yes), the overhead image generation unit 75 uses only the image data determined by the determination unit 74 to use the overhead image. Generate (SA11). For example, an overhead image can be generated by performing a known coordinate conversion process on the image data.

  Then, the generated overhead image is synthesized (SA12), and the synthesized image is stored in a drawing memory (not shown) (SA13). Here, the “composite image” is an image generated by sequentially combining the overhead image generated in SA11 with the composite image already stored in the drawing memory in SA12. For example, the moving distance of the vehicle 2 from the reference position of the composite image recorded in the drawing memory based on information such as the vehicle speed pulse signal input from the vehicle speed sensor 20 and the detected value of the steering angle input from the steering sensor 30. Then, the turning angle is specified, and the specified moving distance and turning angle are associated with the image data and stored in the image memory 76. Then, the overhead view image generated in SA11 based on the specified turning angle is rotated, and the rotated overhead view image is placed at a position corresponding to the moving distance and turning angle of the vehicle 2 in the composite image recorded in the drawing memory. Overwrite and compose. When it is first determined in SA10 that image data is available after the image generation process is activated, the overhead image generation unit 75 generates an overhead image using the image data determined to be available ( SA11), an overhead image is generated using the image data acquired in SA1 and stored in the image memory 76 in SA3. Then, these overhead images are synthesized (SA12), and the synthesized image is stored in the drawing memory (SA13).

  After the composite image is stored in the drawing memory in SA13 or when it is determined in SA10 that the image data is not available (No in SA10), the overhead image generation unit 75 displays the composite image stored in the drawing memory from the display. The part displayed on 60 is extracted, a rotation process is performed according to the turning angle of the vehicle 2, and it displays on the display 60 (SA14). FIG. 5 is a diagram illustrating a composite image. FIG. 5A illustrates a composite image stored in the drawing memory, and FIG. 5B illustrates a composite image displayed on the display 60. . For example, the bird's-eye view image generation unit 75 centers the current position of the vehicle 2 from the composite image (the area indicated by the broken line in FIG. 5A) written in the drawing memory as shown in FIG. The extraction area (rectangular frame shown by the solid line in FIG. 5A) corresponding to the periphery of the vehicle 2 is extracted. Here, since the extracted image is an image oriented in accordance with the turning angle of the vehicle 2 with reference to the reference position in the composite image, the extracted image is set so that the traveling direction of the vehicle 2 is upward. A rotation process is performed, and a plan view when the vehicle 2 is viewed from vertically above is superimposed on the position of the vehicle 2 in the extracted image to generate an image for display as shown in FIG. It is displayed on the display 60. By displaying the composite image extracted and rotated in this manner on the display 60, the distance corresponding to the travel distance of the vehicle 2 (for example, the travel distance per pulse of the vehicle speed pulse signal) each time an image composition trigger is input. Only the composite image in which the image around the vehicle 2 is moved is displayed.

  When it is determined in SA9 that the reference inclination has been updated (SA9, Yes), the determination unit 74 refers to the image memory 76 (SA15), and whether or not each image data stored in the image memory 76 is available. Are sequentially determined (SA16). For example, when the difference between the photographing inclination as the photographing posture stored in the image memory 76 as the header of each image data and the reference inclination is less than the threshold value, it is determined that the image data corresponding to the photographing inclination can be used.

  As a result of the determination in SA16, when it is determined that the image data is usable (SA16, Yes), the overhead image generation unit 75 acquires the image data determined by the determination unit 74 to be usable from the image memory 76, and the acquired image An overhead image is generated using the data (SA17). Then, the generated overhead image is synthesized (SA18), and the synthesized image is stored in a drawing memory (not shown) (SA19). In this case, the bird's-eye view image generation unit 75 determines the position where the oldest image data (image data captured at a position farthest from the current position of the vehicle 2) of the image data determined to be usable by the determination unit 74 is captured. A new reference position for image synthesis is set. Then, the moving distance and turning angle of the vehicle 2 from the newly set reference position are specified for each image data determined to be usable. For example, a newly set reference based on the moving distance and turning angle of the vehicle 2 from the reference position of the composite image before updating the reference inclination stored in the image memory 76 in association with each image data in SA12. The moving distance and turning angle of the vehicle 2 from the position can be specified. Based on the movement distance and turning angle of the vehicle 2 specified in this way, a bird's-eye view image is synthesized. Further, the moving distance and turning angle of the vehicle 2 from the newly set reference position and the image data are stored in the image memory 76 in association with each other. When it is determined in SA9 that the reference inclination has been updated, when an overhead image is first generated in SA17, the overhead image generation unit 75 previously combines the image stored in the drawing memory in the processing of SA13 or the like. The image is deleted, and the overhead image generated in SA17 is stored as it is in the drawing memory as a composite image.

  After the composite image is stored in the drawing memory in SA19, or when it is determined in SA16 that the image data is not available (SA16, No), the determination unit 74 stores all the images stored in the image memory 76 referred to in SA15. It is determined whether it is already determined whether or not the image data can be used (SA20). As a result, if it is determined that it has not been determined whether or not all the image data can be used (SA20, No), the process returns to SA16 and the determination unit 74 can use the next image data stored in the image memory 76. Are determined in order (SA16).

  On the other hand, when it is determined that it has been determined whether or not all the image data can be used (SA20, Yes), the overhead image generation unit 75 finally displays the composite image stored in the drawing memory in SA19 on the display 60. A portion to be extracted is extracted, rotated according to the turning angle of the vehicle 2, and displayed on the display 60 (SA14).

  Thereafter, the processing from SA4 to SA20 is repeated. For example, when the shift position sensor 40 detects that the shift position has been changed to a position other than reverse, or when an instruction to end the image generation process is input via the input unit, the image generation process ends. .

  FIG. 6 is a diagram conceptually showing a configuration of a composite image generated by the image generation processing. FIGS. 6A and 6D show the range of the overhead image generated from the image data and the time of shooting. FIGS. 6B and 6E are diagrams illustrating the relationship, FIGS. 6B and 6E are diagrams illustrating the relationship between the range used for synthesis and the shooting time in each overhead image, and FIGS. 6C and 6F are generated synthesized images. FIG. 6A, 6B, and 6C show a case where image data determined to be usable among the image data acquired by the image acquisition unit 72 is used, and FIGS. 6D and 6E are used. (F) shows a case where all of the image data acquired by the image acquisition unit 72 is used.

  An overhead image is generated from all the image data acquired by the image acquisition unit 72 as shown in FIG. 6D, and the latest overhead is obtained for each range where a plurality of overhead images overlap as shown in FIG. When composition is performed using an image, a composite image as shown in FIG. 6F is generated. In the composite image shown in FIG. 6F, the image is acquired by the image acquisition unit 72, for example, when the difference between the reference inclination and the imaging inclination exceeds a threshold value due to the rear wheel of the vehicle 2 climbing a step. Since the images including the bird's-eye view image generated from the captured image data ("image during step travel" in FIGS. 6D and 6F) are combined, two straight lines drawn parallel to the road surface are formed. Displayed with distortion.

  On the other hand, an overhead image is generated from the image data determined to be usable as shown in FIG. 6A, and the latest overhead image is displayed for each range where a plurality of overhead images overlap as shown in FIG. 6B. When combined, the difference between the reference tilt and the shooting tilt is greater than or equal to the threshold value, and the range corresponding to the image data that was determined to be unusable because the image was shot is used. Is synthesized using a bird's-eye view image generated from image data taken before the difference between the threshold value and the threshold is equal to or greater than the threshold (in FIGS. 6A and 6C, “image during flat road travel before step travel”). Done. Therefore, as shown in FIG. 6C, a composite image is generated in which two straight lines drawn parallel to the road surface are parallel.

(effect)
As described above, according to the first embodiment, based on the difference between the reference posture and the photographing posture, it is determined whether the image data acquired by the image acquisition unit 72 is usable, and the determination unit 74 determines that it is usable. Since the bird's-eye view image of the area including the vehicle 2 is generated using only the image data, the bird's-eye view image can be generated using only the image data without distortion without performing the correction process of the image data. Therefore, regardless of the inclination of the vehicle 2, it is possible to output an image of the periphery of the vehicle 2 without discontinuities or distortion while suppressing an increase in processing load.

  Further, the image data acquired by the image acquisition unit 72 and the shooting posture when the image data is shot are stored in the image memory 76 in association with each other and stored in the image memory 76. When the difference between the photographing inclination as the photographing posture and the reference inclination is less than the threshold value, it is determined that the image data corresponding to the photographing inclination is usable. From this, it is possible to select only available image data without distortion and generate an overhead image.

  Further, when the difference between the shooting inclination specified by the shooting posture specifying unit 73 and the reference inclination is continuously equal to or greater than the threshold while the vehicle 2 moves a predetermined distance, the reference inclination is updated based on the shooting inclination. For example, whether or not image data is available based on an appropriate reference inclination even when the road surface changes, such as when the vehicle 2 enters a sloped road from a flat road Can do.

[Embodiment 2]
Next, a second embodiment will be described. In this mode, only the image data determined to be usable by the determination unit 74 is recorded, and an overhead image is generated using the recorded image data. The configurations of the second embodiment and the subsequent embodiments are substantially the same as the configurations of the first embodiment unless otherwise specified, and this configuration is substantially the same as the configuration of the first embodiment. The same reference numerals and / or names as those used in Embodiment 1 are attached as necessary, and the description thereof is omitted.

(Configuration-Image memory)
First, the configuration of the driving support apparatus according to the second embodiment will be described. The image memory 76 according to the second embodiment is used as a recording unit that records only the image data determined to be usable by the determination unit 74 among the image data acquired by the image acquisition unit 72. As in the first embodiment, the overhead image generated by the overhead image generation unit 75 and the composite image are also recorded.

(Processing-Image generation processing)
Next, an image generation process executed by the driving support device 1 according to the second embodiment will be described. FIG. 7 is a flowchart of the image generation process. Since each process from SC1 to SC4 is the same as each process from SA2 and SA4 to SA6 in FIG. 2 described in the first embodiment, the description thereof is omitted.

After specifying the shooting tilt as the shooting posture in SC4, the reference posture specifying unit 71 executes a reference tilt update process (SC5). Subsequently, the determination unit 74 determines whether or not the image data acquired in SC3 can be used (SC6). For example, when the difference between the photographing inclination specified in SC4 and the reference inclination specified in SC1 or updated in SC4 is less than the threshold value, it is determined that the image data acquired in SC3 can be used.

  As a result, when it is determined that the image data is available (SC6, Yes), the determination unit 74 records the image data in the image memory 76 (SC7). Subsequently, the overhead image generation unit 75 generates an overhead image using the image data recorded in the image memory 76 (SC8). Then, the generated overhead image is synthesized with the synthesized image recorded in the image memory 76 (SC9). Further, a portion to be displayed on the display 60 is extracted from the synthesized composite image, and a rotation process is performed according to the turning angle of the vehicle 2 to display on the display 60 (SC10). Thereafter, the processes from SC2 to SC10 are repeated.

(effect)
As described above, according to the second embodiment, in addition to the same effects as all or part of the first embodiment, the image memory 76 that records only the image data determined to be usable by the determination unit 74 is recorded. Since the overhead image is generated using the existing image data, the capacity of the image data recorded in the image memory 76 can be reduced.

[Modifications to Embodiment]
Although the embodiments of the present invention have been described above, the specific configuration and means of the present invention can be arbitrarily modified and improved within the scope of the technical idea of each invention described in the claims. Can do. Hereinafter, such a modification will be described.

(About problems to be solved and effects of the invention)
First, the problems to be solved by the invention and the effects of the invention are not limited to the above-mentioned contents, and may vary depending on the implementation environment and details of the configuration of the invention. May be solved, or only some of the effects described above may be achieved.

(About judgment part)
In each of the above-described embodiments, the determination unit 74 determines whether or not image data is available based on the difference between the shooting tilt and the reference tilt, but may determine based on other criteria. . For example, lanes and parking frames drawn on the road surface are discontinuous in the image data, or a plurality of straight lines (for example, parking frames) drawn in parallel on the road surface are in the image data. If they are non-parallel, it may be determined that the image data is not usable.

DESCRIPTION OF SYMBOLS 1 Driving assistance apparatus 2 Vehicle 10 Inclination sensor 20 Vehicle speed sensor 30 Steering sensor 40 Shift position sensor 50 Camera 60 Display 70 Control part 71 Reference attitude | position specific | specification part 72 Image acquisition part 73 Image | photographing attitude | position identification part 74 Judgment part 75 Overhead image generation part 76 Image Memory 80 Data recording part

Claims (6)

A reference posture specifying means for specifying a reference posture as a reference for the posture of the vehicle;
Image acquisition means for acquiring image data around the vehicle;
Shooting posture specifying means for specifying a shooting posture that is a posture of the vehicle when shooting of the image data acquired by the image acquisition means is performed;
Determining whether or not the image data acquired by the image acquisition unit is available based on a difference between the reference posture specified by the reference posture specifying unit and the shooting posture specified by the shooting posture specifying unit Means,
An overhead image generation means for generating an overhead image of an area including the vehicle using only the image data determined to be usable by the determination means;
Display means for displaying the overhead image generated by the overhead image generation means;
A driving support apparatus comprising: Image data storage means for storing the image data acquired by the image acquisition means and the shooting posture when the image data is shot in association with each other;
The reference posture specifying means includes
Specifying a reference inclination as a reference of the inclination of the vehicle as the reference posture;
The photographing posture specifying means includes
Specifying the shooting inclination, which is the inclination of the vehicle when the image data acquired by the image acquisition means is shot, as the shooting posture;
The determination means includes
When the difference between the photographing inclination as the photographing posture stored in the image data storage means and the reference inclination is less than a threshold, it is determined that the image data corresponding to the photographing inclination is usable,
The overhead image generation means includes:
Acquiring the image data determined to be usable by the determination unit from the image data storage unit, and generating the overhead image using the acquired image data;
The driving support device according to claim 1. The reference posture specifying means includes
While the vehicle moves a predetermined distance, if the difference between the shooting inclination specified by the shooting posture specifying means and the reference inclination is continuously equal to or greater than the threshold value, the reference inclination is updated based on the shooting inclination. ,
The driving support device according to claim 2. A recording unit that records only the image data determined to be usable by the determination unit;
The overhead image generation means includes:
Generating the overhead image using image data recorded in the recording means;
The driving support device according to claim 1. A reference posture specifying step for specifying a reference posture that is a reference for the posture of the vehicle;
An image acquisition step of acquiring image data around the vehicle;
A shooting posture specifying step for specifying a shooting posture that is a posture of the vehicle when shooting of the image data acquired in the image acquisition step is performed;
Determining whether or not the image data acquired in the image acquisition step is usable based on a difference between the reference posture specified in the reference posture specifying step and the shooting posture specified in the shooting posture specifying step Steps,
An overhead image generation step of generating an overhead image of an area including the vehicle using only the image data determined to be usable in the determination step;
A display step for displaying the overhead image generated in the overhead image generation step;
Driving support method including.   The driving assistance program which makes a computer perform the method of Claim 5.

Images