JP2012040883A - Device for generating image of surroundings of vehicle - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a device for generating an image of surroundings of a vehicle that enables a driver to recognize the situation inside a target parking frame more clearly.SOLUTION: The device 3 for generating the image of surroundings of the vehicle is connectable with a plurality of cameras mounted in the vehicle. The device includes: a parking frame specifying unit 33 for specifying a target parking frame for the vehicle; a captured image selecting unit 34 for selecting, from among the plurality of cameras, at least one camera suitable for capturing an image of the target parking frame on the basis of the specified target parking frame and selecting a captured image from the selected camera; and an additional image drawing unit 35 for generating an image for display, which represents the situation of the specified target parking frame, using the captured image from the selected camera.

Description

  The present invention relates to a vehicle surrounding image generation device that is configured to be connectable to a plurality of in-vehicle cameras and that generates an image that allows a driver of the vehicle to check a situation around the vehicle.

Examples of the vehicle surrounding image generation device as described above include the following parking assistance devices. This parking assistance device uses a vehicle-mounted camera to create an overhead image showing the situation around the vehicle, particularly the situation of the parking frame when looking down from above the vehicle. More specifically, the parking assist device enables the driver to intuitively perform an operation in which the driver specifies the target parking frame, that is, the position and direction of the parking frame from which the vehicle is to be parked. The frame graphic is superimposed on the parking frame position in the overhead image. Thus, the target parking frame is taught to the driver (for example, see Patent Document 1).
JP 2007-230371 A (page 5-7, FIG. 1)

  However, since the three-dimensional object is displayed in a distorted manner in the overhead view image, there is a problem that the driver cannot clearly recognize the three-dimensional object existing in the parking frame even when viewing the displayed overhead image.

  Therefore, an object of the present invention is to provide a vehicle surrounding image generation device that allows a driver to more clearly recognize a situation in a target parking frame.

  In order to achieve the above object, the present invention provides a vehicle surrounding image generation device connectable to a plurality of cameras attached to a vehicle, a parking frame specifying unit that specifies a target parking frame of the vehicle, and a specified target Based on the parking frame, a selection unit that selects at least one camera suitable for photographing the target parking frame from a plurality of cameras, selects a photographed image from the selected camera, and uses a photographed image by the selected camera. And a drawing unit that generates a display image representing the status of the identified target parking frame.

  As described above, the present invention uses the image from the camera selected based on the specified parking frame to generate a display image representing the state of the parking frame. Since the display image uses an image from the selected camera, the distortion of the three-dimensional object that can be reflected in the specified parking frame and / or around the parking frame can be relatively reduced. As described above, according to the present invention, a display image that can reduce the distortion of the three-dimensional object in the parking frame is generated, and when this is displayed, the driver can The situation can be recognized more clearly.

Hereinafter, a vehicle surrounding image display system according to the best mode for carrying out the present invention will be described with reference to the drawings.
(First embodiment)
FIG. 1 is a block diagram showing an overall configuration of a vehicle surrounding image display system 1 according to the first embodiment of the present invention. In FIG. 1, the vehicle surrounding image display system 1 includes, for example, four cameras 2a, 2b, 2c, and 2d, a vehicle surrounding image generation device 3, and a display device 4.

  Each of the cameras 2a to 2d includes a solid-state imaging device such as a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS), and is a digital camera capable of capturing a color image or a monochrome image. Further, these cameras 2a to 2d more preferably have a wide viewing angle (wide angle of view) lens such as a fish-eye lens, for example, and it is preferable that the camera α (see FIG. 2) can be photographed over a wide range. .

  As shown in FIG. 2, the cameras 2 a to 2 d are attached around the vehicle α. As a specific example, in the present embodiment, the camera 2a is attached to the center of the front end of the vehicle α so that the optical axis is directed to the front of the vehicle and the optical axes intersect with each other at a predetermined depression angle. The remaining cameras 2b, 2c, and 2d are attached to the left side, rear center, and right side of the vehicle α toward the left side, rear, and right side of the vehicle. The cameras 2b, 2c, and 2d are also attached so that their optical axes intersect with the road surface at a predetermined depression angle.

  FIG. 2 schematically illustrates a range captured by the cameras 2a to 2d in the peripheral region of the vehicle α. The camera 2a captures the situation of the imaging range CAMa in front of the vehicle α and generates a captured image CPa. The camera 2b captures a captured image CPb by capturing an imaging range CAMa on the left side of the vehicle α, the camera 2c captures a captured image CPc by capturing an imaging range CAMc behind the vehicle α, and the camera 2d A captured image CPd is generated by capturing an imaging range CAMd on the right side of α. These captured images CPa to CPd are exemplified in FIG. 3 and are transmitted to the vehicle surrounding image generation device 3 (see FIG. 1) via a transmission path.

  Here, FIG. 2 will be referred to again. In order to cover the entire periphery of the vehicle α by the cameras 2a to 2d, the imaging range CAMa overlaps the imaging range CAMb. From the same viewpoint, the imaging range CAMb, the imaging range CAMc, the imaging range CAMc, the imaging range CAMd, the imaging range CAMd, and the imaging range CAMa overlap. Thereafter, the overlapping portion of the imaging ranges CAMa and CAMb is the overlap region Fa, the overlapping portion of the imaging ranges CAMb and CAMc is the overlapping region Fb, the overlapping portion of the imaging ranges CAMc and CAMd is the overlapping region Fc, and the imaging range CAMd , CAMa is referred to as an overlap region Fd.

  Here, in order for the vehicle surrounding image generation device 3 to generate various images, the camera parameters representing the positions and orientations (orientations) of the cameras 2a to 2d and the distortion of the lenses of the cameras 2a to 2d are actually measured. Or the like, and is preferably held by the vehicle surrounding image generation device 3. Further, these camera parameters may be calculated using a calibration target (mark).

  Refer to FIG. 1 again. The vehicle surrounding image generation device 3 is an ECU (Electronic Control Unit), for example, and includes a processor, a non-volatile memory, and the like. As shown in FIG. 4, as many input buffers 31a, 31b, 31c, 31d, an overhead image generation unit 32, a parking frame specification unit 33, a captured image selection unit 34, an additional image drawing unit 35, and an output buffer 36. In the present embodiment, the overhead image generation unit 32, the parking frame specification unit 33, the captured image selection unit 34, and the additional image drawing unit 35 are realized by a processor that executes software stored in advance in the nonvolatile memory. Suppose that Such a vehicle surrounding image generation device 3 generates a display image DP output by the display device 4 at the subsequent stage.

  Reference is again made to FIG. A display device 4 is also connected to the vehicle surrounding image generation device 3 via a transmission path. The display device 4 is, for example, a liquid crystal display, and displays the display image DP generated by the vehicle surrounding image generation device 3.

  Next, with reference to FIG. 5, the flow of processing of the vehicle surrounding image display system 1 having the above configuration will be described.

  In FIG. 5, the parking frame specific | specification part 33 receives the start trigger of parking frame detection first (FIG. 5; step S301). As an example of this trigger, there is a case where specific shift position information and operation information are input. The shift position information is information indicating the shift position of the vehicle. For example, when the shift position information indicating the reverse range is input, the parking frame specifying unit 33 determines that the start trigger has been received. The operation information is information indicating that a physical button (not shown) or a graphical button (not shown) provided in the vehicle surrounding image display system 1 is operated by the driver. The parking frame specifying unit 33 determines that a start trigger has been received, for example, when operation information indicating that a parking frame detection button that is one of these buttons has been pressed is input.

  The cameras 2a to 2d are connected to input buffers 31a to 31d as shown in FIG. The captured images CPa to CPd obtained by the cameras 2a to 2d are periodically stored in the input buffers 31a to 31d when the parking frame specifying unit 33 accepts the start trigger at the latest. Following step S301, the parking frame specifying unit 33 acquires the captured images CPa to CPd stored in the input buffers 31a to 31d (step S302).

  Next, the parking frame specifying unit 33 performs target parking frame detection processing (step S304). As an example of the target parking frame detection process, there is a method disclosed in Patent Document 1 (Japanese Patent Laid-Open No. 2007-230371) described above. As an alternative example, the parking frame specifying unit 33 detects a white line on the road surface from the captured images CPa to CPd by performing known image processing on the captured images CPa to CPd acquired in step S302. Furthermore, an area surrounded by white lines or an area sandwiched by white lines is detected as a parking area (hereinafter referred to as a parking frame). As a further alternative, the parking frame identification unit 33 may detect the target parking frame using an active sensor such as a white line region detected by a radar or a near infrared camera that emits near infrared rays.

  Following step S304, the parking frame specifying unit 33 determines whether or not the target parking frame has been successfully detected (step S305).

  If a negative determination is made in step S305, the parking frame specifying unit 33 returns to step S302, acquires the updated captured images CPa to CPd, and performs step S304 again.

  If an affirmative determination is made in step S305, the parking frame specifying unit 33 calculates the relative positional relationship between the host vehicle and the target parking frame (step S306). Specifically, the relative positional relationship is calculated from the detected world coordinates of the target parking frame and the world coordinates of the host vehicle. For example, the imaging ranges CAMa to CAMd (see FIG. 2) that can be captured by the respective cameras are calculated based on the camera parameters described above, and the calculated values are stored in the nonvolatile memory described above. From the world coordinate value that can specify each imaging range CAMa to CAMd stored in this nonvolatile memory, the world coordinate value of the attachment position of each camera 2a to 2d, and the world coordinate value of the detected target parking frame, A relative positional relationship of the target parking frame with respect to the host vehicle is calculated. Here, as the relative positional relationship, the distance and direction of the target parking frame with respect to the host vehicle and the world coordinate values of both are typical.

  Next to step S306, the parking frame specifying unit 33 determines whether or not the bird's-eye view image is better generated this time by the vehicle surrounding image generation device 3 (step S307). In this embodiment, since an overhead image or a captured image is generated, if the determination is negative in step S307, a captured image is generated. In addition, the distance from the host vehicle to the target parking frame is used as the determination criterion in step S307. For example, if this distance is less than a predetermined threshold, that is, if the distance is relatively close, an overhead image is good. In addition, if the distance is equal to or greater than the same threshold value, that is, if the distance is relatively long, it is programmed in advance so that the camera image is determined to be good.

  If an affirmative determination is made in step 307, the parking frame identification unit 33 gives an instruction to generate an overhead image to the overhead image generation unit 32 in step S308. In response to this instruction, the overhead image generation unit 32 acquires the captured images CPa to CPd stored in the input buffers 31a to 31d, and performs geometric conversion on the acquired captured images CPa to CPd. As a result, an overhead image representing the situation when looking down around the host vehicle is generated from a virtual viewpoint preset above the host vehicle (step S308). Since the processing for generating the bird's-eye view image is well known, detailed description thereof is omitted.

  Here, FIG. 6 is a schematic diagram illustrating an example of the bird's-eye view image generated in step S308. In FIG. 6, the bird's-eye view image LDP is synthesized with a model image Mα that imitates the host vehicle at a predetermined position. Further, a road surface RS (see the hatched portion) and white lines WL indicating some parking frames PL are drawn around the model image Mα. In FIG. 6, typically, one parking frame PL is virtually shown by a one-dot chain line, and one other vehicle Va is also shown. The overhead image as described above is passed to the additional image drawing unit 35.

  Reference is again made to FIG. If a negative determination is made in step S307 described above, the parking frame specifying unit 33 determines in the cameras 2a to 2d based on the relative positional relationship of the target parking frame with respect to the host vehicle (calculated in step S306) in step S309. Therefore, the closest camera to the target parking frame successfully detected in step S304 is selected as the optimum camera. Thereafter, the parking frame specifying unit 33 passes information indicating the optimum camera to the captured image selecting unit 34. In response to this information, the captured image selection unit 34 captures images stored in an input buffer (any one of the input buffers 31a to 31d) connected to the optimum camera (any one of the cameras 2a to 2d). The image is acquired as the optimum captured image and is passed to the additional image drawing unit 35 (step S309).

Here, FIG. 7 is a schematic diagram illustrating an example of the optimum captured image selected in step S309. In the example of FIG. 7, the optimal captured image ATP is an image captured from the viewpoint of the optimal camera,
At least a road surface RS (see the hatched portion), a white line WL indicating the target parking frame TPL, and a portion α of the host vehicle are drawn. In FIG. 7, the target parking frame TPL is virtually drawn with a one-dot chain line.

  When the above step S308 or S309 is completed, the parking frame specifying unit 33 passes an instruction to draw various predetermined additional images to the additional image drawing unit 35 in step S310. The relative positional relationship of the target parking frame TPL is also passed to this instruction. In response to this instruction, as shown in FIGS. 8A and 8B, the additional image drawing unit 35 adds the target parking as an example of the additional image to the input overhead image LDP or the optimum captured image ATP. A display image representing at least the status of the target parking frame TPL by superimposing predetermined ones among the frame image FP, the mask image MKP (see the hatched portion), the icon ICN, and the graphic operation button GB. A DP is generated (step S310).

  In this embodiment, the target parking frame image FP is, for example, a rectangular frame graphic for indicating the target parking frame TPL, and the superimposed position is a relative positional relationship sent from the parking frame specifying unit 33. And it is specified by camera parameters stored in advance. In this embodiment, the mask image MKP is an image for covering an unnecessary part (for example, a part that is not desired to be displayed) in the overhead view image or the optimum captured image. Further, the icon ICN is a graphic indicating in which direction the target parking frame TPL is with respect to the host vehicle. The operation button GB is, for example, a button for re-detecting a parking frame, and is operated by a driver, and various functions are assigned in advance. Mask image MKP, icon ICN, and operation button GB are superimposed on a predetermined position with respect to the input overhead image or the input optimum captured image.

  The overhead view image or the optimum captured image on which various additional images are superimposed is output to the output buffer 36 (see FIG. 4) as the display image DP. The display device 4 receives and displays the display image DP from the output buffer 36 (step S311).

  The process of the vehicle surrounding image display system 1 shown in FIG. 1 is repeatedly performed.

  Here, with reference to FIG.9 and FIG.10, the detailed process of step S307 is demonstrated. FIG. 9 is a schematic diagram showing the positional relationship between the target parking frame and the host vehicle. FIG. 9 shows an actual target parking frame TPL, the host vehicle α, and a drawing area LDA (see an area surrounded by a two-dot chain line) drawn as an overhead image. A drawing area LDA having a predetermined area is set in the vehicle surrounding display system 1 from the world coordinate positions of the target parking frame TPL and the host vehicle α. In the drawing area LDA, as shown in FIG. 9A, when a part or the whole of the target parking frame TPL is included, in step S307, when the overhead image is generated, the parking frame specifying unit 33 judge. On the other hand, if the drawing area LDA does not include a part or the whole of the target parking frame TPL as shown in FIG. The frame specifying unit 33 determines.

  As the above reason, it is generally known that in a bird's-eye view image, a three-dimensional object drawn there is distorted as if it falls on the road surface. FIG. 10 is a schematic diagram showing the fall of the three-dimensional object MMS, which is correlated with the distance from the camera 2b, as an example of such a fall. As shown in FIG. 10, on the overhead image, the three-dimensional object MMS is drawn in a state of being projected on the road surface RS. Further, the mounting height itself of the camera 2b is not substantially changed. Accordingly, the distortion changes depending on the positional relationship between the three-dimensional object MMS and the camera 2b, and the distortion (length) Lf of the three-dimensional object MMS increases as the distance between the two increases. That is, when the target parking frame TPL is close, even if the overhead image is displayed on the display device 4, the three-dimensional object that is drawn in the overhead image and can exist around the target parking frame TPL has only a small distortion. Does not occur. From the above viewpoint, when the target parking frame TPL exists nearby, the overhead image LDP is used as the display image DP as shown in FIG. On the contrary, when the target parking frame TPL exists far away, as shown in FIG. 8B, the optimum captured image ATP is used for the display image DP.

  Next, the optimum camera selection criteria in step S309 will be described with reference to FIG. FIG. 11 is a schematic diagram showing selection criteria for the optimum captured image based on the target parking frame TPL and the positional relationship between the cameras 2a to 2d. As shown in FIGS. 11A to 11D, the determination of which camera's captured image is used is based on the world coordinate values that define the imaging ranges CAMa to CAMd (see FIG. 2) of the cameras 2a to 2d. The parking frame specifying unit 33 holds a type of camera parameter in advance. And since the world coordinate value of the target parking frame TPL is obtained in step S305, the parking frame specifying unit 33 determines which imaging range the target parking frame TPL is in the imaging ranges CAMa to CAMd, and the target parking frame A camera having an imaging range including the frame TPL is selected as the optimum camera.

  Here, as shown in FIG. 2, in the present embodiment, the cameras 2 a to 2 d are installed with respect to the vehicle α so that the imaging ranges of adjacent cameras overlap in the overlap regions Fa to Fd. Some or all of the target parking frame TPL may be included in these overlap areas Fa to Fd. In such a case, as shown in FIG. 12, the parking frame specifying unit 33 selects the optimum camera. In FIG. 12, it is assumed that the target parking frame TPL (see the portion with dots) is in the overlap region Fa (see the lattice-shaped hatching portion). Under this assumption, as shown in FIG. 12, the parking frame specifying unit 33 selects one of the cameras 2a and 2b as the optimum camera. The following two methods are exemplified as the selection method. In the first method, as shown in FIG. 12A, distances Da and Db between a point closest to the vehicle α in the target parking frame TPL and any two of the cameras 2a to 2d are obtained. The camera having the smaller one of Da and Db is selected as the optimum camera. In the second method, first, a point closest to the vehicle α in the target parking frame TPL and two lines La and Lb connecting the cameras 2a and 2b are obtained. Further, an angle θa formed by the line La and the optical axis Xa of the camera 2a and an angle θb formed by the line Lb and the optical axis Xb of the camera 2b are obtained. Thereafter, the camera having the smaller one of the two angles θa and θb is selected as the optimum camera.

  Above, description of this embodiment is complete | finished. As described above, in the vehicle surrounding image display system 1, the parking frame specifying unit 33 specifies the target parking frame TPL, and then obtains the relative positional relationship of the target parking frame TPL with respect to the host vehicle α. Further, when the target parking frame TPL is relatively far from the own vehicle α, the parking frame specifying unit 33 selects the camera most suitable for showing the specified target parking frame TPL to the driver, and is selected by this. A display image is created using a photographed image from the optimum camera. As described above with reference to FIG. 10, the captured image displayed on the display device 4 has less distortion of the three-dimensional object around the target parking frame TPL than the overhead image. Thus, according to the vehicle surrounding image display system 1, the driver selects the optimum camera in accordance with the positional relationship with the target parking frame TPL. It becomes possible to recognize this situation more clearly.

  In the above embodiment, when the target parking frame TPL exists far from the host vehicle α, the display image is created using the captured image itself. However, the present invention is not limited to this, and one or a plurality of captured images may be used as a display image after converting the viewpoint to an image when the target parking frame TPL is viewed from the viewpoint of the driver, for example.

Moreover, in the above embodiment, when the target parking frame TPL exists far from the host vehicle α, the optimum captured image ATP is used for the display image DP, and the target parking frame TPL exists nearby. In this case, the overhead image LDP is used as the display image DP. However, the present invention is not limited to this, and when the target parking frame TPL exists in the distance, as shown in FIG. 13, a composite image of both the overhead image LDP and the optimum captured image ATP is created as the display image DP. It doesn't matter.
(Second Embodiment)
FIG. 14 is a block diagram showing an overall configuration of a vehicle periphery image display system 1a according to the second embodiment of the present invention. The vehicle surrounding image display system 1a in FIG. 14 is different from the vehicle surrounding image display system 1 shown in FIG. 1 in that the vehicle surrounding image generation device 3 is replaced with the vehicle surrounding image generation device 3a. Since the other configurations are the same, in the following, the components corresponding to those in FIG. 1 are denoted by the same reference numerals, and detailed descriptions thereof are omitted.

  FIG. 15 is a block diagram showing a detailed configuration of the vehicle surrounding image generation device 3a shown in FIG. The vehicle surrounding image generation device 3a in FIG. 15 is different from the vehicle surrounding image generation device 3 in terms of functional blocks in that an obstacle detection unit 36 is added as shown in FIG. Since the other configurations are the same, in the following, in FIG. 15, the components corresponding to the configurations in FIG. 4 are denoted by the same reference numerals, and detailed descriptions thereof are omitted.

  The obstacle detection unit 36 is realized by, for example, a processor that executes software stored in advance in the nonvolatile memory. The obstacle detection unit 36 receives the captured images CPa to CPd and the relative positional relationship from the parking frame specifying unit 33 from the input buffers 31 a to 31 d. Thereafter, the obstacle detection unit 36 detects an obstacle (a three-dimensional object) that may exist around the target parking frame TPL. When the obstacle is successfully detected, the obstacle detection unit 36 passes the world coordinate value where the detected obstacle exists to the additional image drawing unit 35.

  Next, with reference to FIG. 16, the process flow of the vehicle surrounding image display system 1a configured as described above will be described. FIG. 16 differs from FIG. 5 in that steps S601, S602, and S603 are added instead of step S310. Since the other steps are the same, in the following, the steps corresponding to the steps in FIG. 5 are given the same step numbers, and detailed descriptions thereof are omitted.

  When step S308 or S309 ends, the obstacle detection unit 37 performs obstacle detection (step S601). An obstacle is a stationary object such as a wall existing in a parking lot, an already parked vehicle, or a pillar, or a moving object such as a moving person, another vehicle, or a bicycle.

  For example, the obstacle detection unit 37 holds the captured images CPa to CPd obtained from the input buffers 31a to 31d for the past several frames. The obstacle detection unit 37 detects a moving object such as a well-known optical flow by using the captured images CPa for the past several frames from the present to detect a moving object, or 2 having a time difference such as motion stereo. A still object is detected using the stereo principle using the captured image CPa for the frame. Similarly, moving objects and stationary objects are detected for the other captured images CPb to CPd.

  Here, preferably, the obstacle detection unit 37 obtains which one of the optimum cameras selected by the parking frame specifying unit 33 in order to reduce the processing load, and obstructs the optimum captured image obtained from the optimum camera. It is preferable to perform object detection.

  Next, when the obstacle detection unit 37 succeeds in detecting the obstacle in step S601, the obstacle detection unit 37 obtains a world coordinate value that specifies the position of the detected obstacle. If the obstacle is a moving object, the obstacle detecting unit 37 approaches the target parking frame TPL detected in step S304 from the obtained world coordinate value and information on the moving direction of the moving object. It is determined whether the vehicle α is approaching in the moving direction. This determination method will be described in detail later. Further, the obstacle detection unit 37 also obtains the moving speed of the moving object.

  The obstacle detection unit 37 passes the world coordinate values, movement direction, and movement speed of the obstacle obtained as described above to the additional image drawing unit 35. Thus, the determination process in step S602 ends.

  The additional image drawing unit 35, based on the information received from the obstacle detection unit 37, in addition to the processing of step S310 described above, indicates that a stationary object is present near the target parking frame TPL, and that the moving object is the target parking object TPL or An obstacle icon indicating that the vehicle α is approaching the future course is superimposed on the corresponding position in the overhead image LDP or the optimum captured image ATP in the display image DP (step S603).

  Here, FIG. 17A is a schematic diagram showing an example of the display image DP using the overhead image LDP and further superimposed with the obstacle icon OBS, and FIG. 17B shows the optimum captured image ATP. It is a schematic diagram which shows an example of the image DP for display on which the obstacle icon OBS was further superimposed. 17 (a) and 17 (b) are different from the display image DP shown in FIGS. 8 (a) and 8 (b) in that an obstacle icon OBS is superimposed, and thus description thereof is omitted. . In the present embodiment, the bird's-eye view image LDP depicts a portion at a short distance from the vehicle α in the peripheral region of the target parking frame TPL. However, it is preferable that the obstacle icon OBS is drawn on the display image DP in some form even in the peripheral region of the target parking frame TPL or when the obstacle is at a long distance from the vehicle α. Considering this, in this embodiment, the obstacle icon OBS can be drawn on the mask image MKP as shown in FIG.

  Here, the additional image drawing unit 35 may hold a table 351 as shown in FIG. 18 in the aforementioned nonvolatile memory. In FIG. 18, a table 351 describes an information set of the color of the icon OBS, the moving speed range of the obstacle, and the distance range from the host vehicle α to the obstacle for each approach degree. In the illustrated example, when the approach degree is level 1, the color of the icon OBS is described as red so that the driver can be most alerted, the moving speed range is described as V2 <Vth ≦ V3, and the distance range is , 0 <Dth ≦ D1. Further, when the degree of approach is level 2, the color of the icon OBS is described as yellow so that the driver can be alerted moderately, the moving speed range is described as V1 <Vth ≦ V2, and the distance range is It is described as D1 <Dth ≦ D2. Further, when the degree of approach is level 3, the color of the icon OBS is described as blue so that the driver can be most gently alerted, the moving speed range is described as 0 <Vth ≦ V1, and the distance range is It is described as D2 <Dth ≦ D3.

  With reference to such a table 351, the additional image drawing unit 35 preferably changes the color of the above-described obstacle icon OBS based on the information received from the obstacle detection unit 37. Further, the color of the target parking frame image FP may be changed to the same color as the obstacle icon OBS.

  Next, with reference to FIG. 19, the above approach degree determination method will be described in detail. FIG. 19 is a schematic diagram illustrating a method for determining the degree of approach when there is an obstacle X that moves near the target parking frame TPL. In addition to the target parking frame TPL and the obstacle X, FIG. 19 shows the host vehicle α and a vector V indicating the moving speed of the obstacle X. As described above, the world coordinate value of the target parking frame TPL is known, and the vector V indicating the moving speed and moving direction of the obstacle X is also calculated by the obstacle detecting unit 37. The additional image drawing unit 35 can first determine whether the obstacle X is approaching or not approaching the target parking frame TPL.

  If it is determined that the obstacle X is approaching the target parking frame TPL, then the additional image drawing unit 35 causes the current movement speed of the obstacle X to fall within the level 1 movement speed range shown in FIG. If it enters, the color of the obstacle icon OBS is determined to be red, and if it enters the level 2 movement speed range, the color of the obstacle icon OBS is determined to be yellow, and if it enters the level 3 movement speed range For example, the color of the obstacle icon OBS is determined to be blue. For example, a display image DP as shown in FIGS. 17A and 17B is created using the determined color.

  In the above description, a moving object as an obstacle has been described as an example. However, the additional image drawing unit 35 is similar to a stationary object, and calculates the distance between the position of the detected stationary object and the target parking frame TPL. The color of the obstacle icon OBS is determined according to the table 351 of FIG. 18 and is superimposed on the corresponding position in the overhead image LDP or the optimal captured image ATP in the display image DP. Also in this case, the color of the target parking frame TPL may be drawn in the same color as the obstacle icon OBS.

  An obstacle may be detected using an active sensor such as a near-infrared camera by irradiating radar or near-infrared rays.

  The vehicle surrounding image generation device according to the present invention is useful for a parking assistance device, a driving assistance device, and the like that are required to allow the driver to recognize the situation in the target parking frame more clearly.

1 is a block diagram showing an overall configuration of a vehicle surrounding image display system 1 according to a first embodiment of the present invention. The cameras 2a to 2d shown in FIG. Schematic diagram showing an example of captured images CPa to CPd from the cameras 2a to 2d shown in FIG. The block diagram which shows the detailed structure of the vehicle surrounding image generation apparatus 3 shown in FIG. The flowchart which shows the flow of a process of the vehicle surrounding image display system 1 shown in FIG. Schematic diagram showing an example of the overhead image LDP generated in step S308 of FIG. FIG. 5 is a schematic diagram showing an example of the optimum captured image ATP obtained in step S309 in FIG. FIG. 5 is a schematic diagram illustrating an example of the display image DP generated in step S310 of FIG. Schematic diagram showing the positional relationship between the target parking frame TPL and the vehicle α Schematic diagram showing the collapse of the three-dimensional object MMS correlated with the distance from the camera 2b Schematic diagram showing selection criteria for the optimum camera in step S309 The schematic diagram which illustrates the selection method of the optimal camera in case the target parking frame TPL exists in an overlap area | region FIG. 5 is a schematic diagram showing an alternative example of the display image DP generated in step S310 of FIG. The block diagram which shows the whole structure of the vehicle periphery image display system 1a which concerns on the 2nd Embodiment of this invention. The block diagram which shows the detailed structure of the vehicle surrounding image generation apparatus 3a shown in FIG. The flowchart which shows the flow of a process of the vehicle surrounding image display system 1a shown in FIG. Schematic diagram showing an example of a display image DP on which an obstacle icon OBS is superimposed FIG. 15 is a schematic diagram showing a table 351 held by the additional image drawing unit 35 shown in FIG. The schematic diagram which shows the determination method of the approach degree in case the obstruction X which moves near the target parking frame TPL exists

Explanation of symbols

1, 1a Vehicle surrounding image display systems 2a, 2b, 2c, 2d Cameras 3, 3a Vehicle surrounding image generating devices 31a, 31b, 31c, 31d Input buffer 32 Overhead image generating unit 33 Parking frame specifying unit 34 Captured image selecting unit 35 Addition Image drawing unit 36 Output buffer 37 Obstacle detection unit 4 Display device

Claims (10)

A vehicle surrounding image generation device connectable to a plurality of cameras attached to a vehicle,
A parking frame specifying unit for specifying the target parking frame of the vehicle;
A selection unit that selects at least one camera suitable for photographing the target parking frame from the plurality of cameras based on the identified target parking frame, and selects a captured image from the selected camera;
A drawing unit that generates a display image representing a situation of the specified target parking frame using a photographed image by the selected camera;
A vehicle surrounding image generation device comprising: The vehicle surrounding image generation device further includes an overhead image generation unit that generates an overhead image that represents a situation when the vehicle periphery of the vehicle is viewed from a predetermined virtual viewpoint using images captured by the plurality of cameras.
After the parking frame specifying unit specifies the target parking frame, the drawing unit generates a display image using a bird's-eye view image, or generates a display image using an image captured by the selected camera. Determine whether
The drawing unit generates the display image according to a determination result of the parking frame specifying unit.
The vehicle surrounding image generation device according to claim 1. The parking frame specifying unit generates a display image using the overhead view image based on a relative positional relationship between the vehicle and the target parking frame, or an image captured by the selected camera. To determine whether to generate a display image,
The vehicle surrounding image generation apparatus according to claim 2. The relative positional relationship is a distance between the vehicle and the target parking frame,
The parking frame specifying unit determines that the drawing unit generates a display image using an overhead image when the distance is less than a predetermined threshold value, and when the distance is equal to or greater than the predetermined threshold value, shooting by the selected camera. It is determined to generate an image for display using the image,
The vehicle surrounding image generation device according to claim 3. The drawing unit generates a display image representing the status of the specified target parking frame by using a captured image obtained by the selected camera without converting the viewpoint.
The vehicle surrounding image generation device according to claim 1. The drawing unit generates a display image representing the status of the identified target parking frame by using a viewpoint image of the image captured by the selected camera.
The vehicle surrounding image generation device according to claim 1. The drawing unit superimposes a predetermined one of a mask image, an icon, and a graphic operation button around a photographed image of the selected camera,
The mask image is used as a mask for the captured image, the icon indicates in which direction the target parking frame is located with respect to the vehicle, and the operation button is operated by a driver of the vehicle.
The vehicle surrounding image generation device according to claim 1. The vehicle surrounding image generation device further includes a three-dimensional object detection unit that detects an obstacle existing in or around the specified target parking frame,
The drawing unit
When an obstacle is detected in or around the specified target parking frame by the three-dimensional object detection unit, an obstacle icon indicating that is superimposed on the display image.
The vehicle surrounding image generation device according to claim 1. A display device that outputs the display image is connected to the vehicle surrounding image generation device.
The vehicle surrounding image generation device according to claim 1. A vehicle surrounding image display system,
A plurality of cameras attached to the vehicle;
A parking frame specifying unit for specifying the target parking frame of the vehicle;
A selection unit that selects at least one camera suitable for photographing the target parking frame from the plurality of cameras based on the identified parking frame;
A drawing unit that generates a display image representing a situation of the specified target parking frame using a photographed image by the selected camera;
A display device for outputting the generated display image;
A vehicle surrounding image display system comprising: