JP5136256B2 - Parking assist device and image display method - Google Patents

Description

  The present invention relates to a parking assistance device and an image display method for supporting a parking operation of a host vehicle by displaying an overhead image around the host vehicle on a display unit.

2. Description of the Related Art Conventionally, a technique is known in which a plurality of cameras mounted on a host vehicle capture images of the vicinity of the host vehicle, convert the obtained image into a bird's-eye view image, and display it on a display (for example, see Patent Document 1) .) The bird's-eye view image is displayed on the display as an image looking down from the virtual viewpoint above the vehicle, so it is easy to objectively grasp the position and distance of the vehicle with respect to the white line on the road surface. This is an effective image for supporting the operation.
JP 2003-169323 A

  However, the overhead image is an image generated by converting the viewpoint of the image captured by the camera with the road surface as the projection plane, and thus the three-dimensional object with a height is displayed as an image that falls down in a direction away from the camera. . For this reason, when a bird's-eye view image is displayed on the display and the parking operation of the host vehicle is supported, if there is a three-dimensional object in the vicinity of the target parking position that is the position where the host vehicle is parked, the three-dimensional object Falls to the target parking position side and is displayed on the display, which makes it difficult to recognize the target parking position on the image.

  The present invention was devised in view of the conventional problems as described above, and displays an image that can accurately recognize the target parking position even when there is a three-dimensional object in the vicinity of the target parking position. An object of the present invention is to provide a parking assistance device and an image display method that can appropriately support the parking operation of the host vehicle.

  The present invention provides a target parking position that is a position where a vehicle is parked on an overhead image when an image around the own vehicle imaged by an imaging means mounted on the own vehicle is converted into a viewpoint image and displayed on the display means. Recognize the position, detect a three-dimensional object in the vicinity of the target parking position, and based on the relationship between the position of the vehicle, the target parking position and the position of the three-dimensional object, the image of the three-dimensional object is displayed on the overhead image. By determining whether or not to fall to the target parking position side, when it is determined that the image of the three-dimensional object falls to the target parking position side, by compressing the image of the three-dimensional object on the overhead image and displaying it on the display means, The problem is solved.

  According to the present invention, when the image of the three-dimensional object falls on the target parking position side on the overhead image, the three-dimensional object image on the overhead image is compressed and displayed on the display means. Even when there is a three-dimensional object, the target parking position can be accurately recognized on the overhead image regardless of the position of the vehicle.

  Hereinafter, as a specific embodiment of the present invention, a function of generating and displaying a bird's-eye view image (hereinafter referred to as an all-around bird's-eye view image) in which the entire periphery of the vehicle is viewed from a virtual viewpoint above the vehicle is described. An example in which the present invention is applied to a parking assist apparatus will be described with reference to the drawings.

  FIG. 1 is a block diagram illustrating a schematic configuration of the parking assist device according to the present embodiment. The parking assist device of the present embodiment is an image processing device 10 that uses all images captured by four onboard cameras, a front camera 1, a rear camera 2, a right side camera 3, and a left side camera 4 mounted on the vehicle. A surrounding bird's-eye view image is generated, and this all-around bird's-eye view image is displayed on the display 5 installed in the cabin of the own vehicle to support the parking operation of the own vehicle. The in-vehicle cameras 1 to 4 are connected, and the display 5 is connected to the output side of the image processing apparatus 10. The image processing apparatus 10 is connected to an operation input device 6 that receives an operation input from the driver of the host vehicle.

  The four in-vehicle cameras including the front camera 1, the rear camera 2, the right side camera 3, and the left side camera 4 are, for example, wide-angle CCD cameras or CMOS cameras having an angle of view of about 180 degrees. It is attached to the front, rear, left and right of the own vehicle so that images of all areas surrounding the own vehicle can be taken. Specifically, as shown in FIG. 2, the front camera 1 is installed, for example, on a front grill or the like in front of the host vehicle, and captures an image in a direction in which the area in front of the host vehicle is looked down obliquely with respect to the road surface. The rear camera 2 is installed in, for example, a roof spoiler behind the host vehicle and captures an image in a direction in which the area behind the host vehicle is looked down obliquely with respect to the road surface. The right side camera 3 is installed, for example, on the right side mirror of the own vehicle and captures an image in a direction in which the area on the right side of the own vehicle is looked down obliquely with respect to the road surface. Further, the left side camera 4 is installed, for example, on the left side mirror of the own vehicle and captures an image in a direction in which the area on the left side of the own vehicle is looked down obliquely with respect to the road surface. These four in-vehicle cameras 1 to 4 capture images of each area around the vehicle at a predetermined imaging cycle (for example, 100 msec). These images are input to the image processing apparatus 10 as needed.

  The display 5 displays an all-around overhead image generated by the image processing device 10 and presents it to the driver of the host vehicle. For example, a liquid crystal display device or the like is used. The display 5 is installed at a position that is easy to see for the driver of the vehicle, such as a center console in the vehicle interior of the vehicle, and is generated by the image processing device 10 based on an image signal output from the image processing device 10. Display an all-around overhead image.

  The operation input device 6 receives an operation input by the driver of the own vehicle, and various operation buttons, a joystick, etc. are used, for example. Further, a touch panel may be provided on the display screen of the display 5, and this touch panel may be used as the operation input device 6. An operation input by the driver of the own vehicle using the operation input device 6 is input to the image processing apparatus 10.

  The image processing apparatus 10 includes a microcomputer that operates according to a predetermined image processing program, a frame memory, and the like. The image processing apparatus 10 inputs an image captured by the in-vehicle cameras 1 to 4 to generate an all-around overhead image, and the image signal Is output to the display 5. The processing by the image processing apparatus 10 is performed in accordance with the imaging cycle of the images of the in-vehicle cameras 1 to 4, and the latest all-around bird's-eye view is obtained by the image processing apparatus 10 every time a new image is input from the in-vehicle cameras 1 to 4. An image is generated, and the image signal is output to the display 5. Therefore, the all-around overhead image generated by the image processing device 10 is displayed on the display 5 as a moving image while being updated as needed. Note that the image processing apparatus 10 can also be configured using an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or the like in which each processing function is incorporated as a circuit instead of the microcomputer.

  When the image processing apparatus 10 grasps the internal configuration functionally, as illustrated in FIG. 3, as illustrated in FIG. 3, the viewpoint conversion unit 11, the target parking position recognition unit 12, the predicted trajectory calculation unit 13, and the three-dimensional object detection unit 14. And an image storage unit 15 and a display control unit 16.

  The viewpoint conversion unit 11 includes an image of a region in front of the vehicle input from the front camera 1, an image of a region in the rear of the vehicle input from the rear camera 2, and a region on the right side of the vehicle input from the right side camera 3. The viewpoint conversion processing is performed on the left side camera 4 and the image of the area on the left side of the vehicle input from the left side camera 4 to project an overhead image with the road surface in front of the vehicle as the projection surface and the road surface behind the vehicle. Four overhead images are generated: a bird's-eye view image as a plane, a bird's-eye view image with a road surface on the right side of the vehicle as a projection plane, and a bird's-eye view image with a road surface on the left side of the vehicle as a projection surface. In this viewpoint conversion process by the viewpoint conversion unit 11, the correspondence between the pixels in the images before and after conversion uniquely specified from the lens characteristics, mounting positions, angles, and the like of the in-vehicle cameras 1 to 4 is held as a conversion table. This can be realized by coordinate conversion of each pixel on the image using this conversion table.

  In addition, the viewpoint conversion unit 11 connects the four overhead images generated from the captured images of the four in-vehicle cameras 1 to 4 as described above, and as shown in FIG. Is generated from a virtual viewpoint above the own vehicle. In the all-around bird's-eye view image shown in FIG. 4, the image area A1 is a bird's-eye view image generated by converting the viewpoint of the image captured by the front camera 1, and the image area A2 is generated by viewpoint-converting the image captured by the rear camera 2. It is a bird's-eye view image, the image area A3 is a bird's-eye view image generated by viewpoint-converting the image captured by the right side camera 3, and the image area A4 is a bird's-eye view image generated by viewpoint-converting the image captured by the left side camera 4. . The vehicle-shaped image in the center of the image is a virtual vehicle image representing the vehicle position, and is an image superimposed with an image created by computer graphics or the like.

  The target parking position recognition unit 12 recognizes a target parking position that is a position where the host vehicle is parked on the all-around overhead image generated by the viewpoint conversion unit 11. Specifically, the target parking position recognition unit 12 performs, for example, a known white line recognition process on the all-around bird's-eye view image generated by the viewpoint conversion unit 11, and with a space equivalent to the vehicle width around the own vehicle. When a pair of white lines arranged in parallel is recognized, an area between the pair of white lines is recognized as a target parking position. In addition, when the driver of the own vehicle sets a parking frame at an arbitrary position on the all-around bird's-eye view image using the operation input device 6, the target parking position recognition unit 12 sets the parking set by the driver's operation input. You may make it recognize the area | region enclosed with a frame as a target parking position.

  The predicted trajectory calculation unit 13 calculates a predicted trajectory until the host vehicle reaches the target parking position based on the positional relationship between the target parking position recognized by the target parking position recognition unit 12 and the current position of the host vehicle. Specifically, the predicted trajectory calculation unit 13 assumes that the host vehicle is parked while traveling backward to the target parking position, for example, and the host vehicle reaches a position that can be reached by backward traveling without turning back to the target parking position. When there is, an arc or straight line connecting the current position of the vehicle and the target parking position is calculated as a predicted trajectory. In addition, when the vehicle is in a position that needs to be turned back to reach the target position by reverse travel, the arc or straight line connecting the current position of the vehicle and the turn-back position, and the turn-back position and the target parking position are set. A connecting arc or straight line is calculated as a predicted trajectory.

  The three-dimensional object detection unit 14 detects a three-dimensional object existing in the vicinity of the target parking position recognized by the target parking position recognition unit 12. Specifically, the three-dimensional object detection unit 14, for example, in the vicinity of the target parking position recognized by the target parking position recognition unit 12 in the all-around overhead image generated by the viewpoint conversion unit 11 (for example, centering on the target parking position). When a known pattern recognition process is performed on a surrounding 5m) image area, and a pattern that matches or is similar to a pre-stored three-dimensional object pattern such as a vehicle shape pattern is detected, the vicinity of the target parking position It is detected that there is a three-dimensional object. In addition, since it is known that the shape of a three-dimensional object on the bird's-eye view image generally changes as the vehicle moves, the three-dimensional object detection unit 14 is, for example, near the target parking position in the all-around bird's-eye view image. In the case where the shape of the object detected in the image area changes greatly as the vehicle moves, the presence of a three-dimensional object in the vicinity of the target parking position may be detected.

  The image storage unit 15 is an image of the target parking position in the all-around bird's-eye view image generated by the viewpoint conversion unit 11, particularly when the three-dimensional object image detected by the three-dimensional object detection unit 14 is not collapsed to the target parking position side. An image of the target parking position is stored. The all-around overhead image generated by the viewpoint conversion unit 11 is updated each time a new image is input from the in-vehicle cameras 1 to 4 as described above, but the image of the three-dimensional object detected by the three-dimensional object detection unit 14 is updated. The image of the target parking position when not falling to the target parking position side is stored and held in the image storage unit 15.

  The display control unit 16 shows the parking frame indicating the target parking position recognized by the target parking position recognition unit 12 and the predicted trajectory calculated by the prediction trajectory calculation unit 13 on the all-around bird's-eye view image generated by the viewpoint conversion unit 11. A locus line or the like is drawn and displayed on the display 5. Further, the display control unit 16 has a function of adjusting the all-around bird's-eye view image generated by the viewpoint conversion unit 11 to an image that can be easily viewed by the driver of the own vehicle as necessary, and displaying the image on the display 5. As a characteristic function related to the present invention, when a three-dimensional object in the vicinity of the target parking position is detected by the three-dimensional object detection unit 14, the three-dimensional object image is displayed on the all-around overhead image generated by the viewpoint conversion unit 11. When it is determined that the three-dimensional object image falls to the target parking position side on the all-around overhead image, the three-dimensional object image is superimposed on the target parking position. It has a function (image adjusting means) for compressing it to a size that does not become necessary and displaying it on the display 5.

  Hereinafter, the outline of the image adjustment processing by the display control unit 16 will be specifically described with reference to a parking scene as shown in FIG.

  The example of FIG. 5 is an example in the case where the host vehicle V is parked at the target parking position S between the parked vehicle VP1 and the parked vehicle VP2. In order to be able to park at the target parking position S by reverse traveling, the host vehicle V reaches the turn-back position C via the positions A and B in FIG. While moving to the target parking position S. In the parking scene shown in FIG. 5, an example of an all-around overhead image generated by the viewpoint conversion unit 11 when the vehicle V passes the position A is shown in FIG. An example of the all-around bird's-eye view image generated by the viewpoint conversion unit 11 when passing through is shown in FIG. 6B, and the all-around bird's-eye view image generated by the viewpoint conversion unit 11 when the vehicle V reaches the position C. Examples of these images are shown in FIG. In addition, all of the image examples in FIGS. 6A to 6C show a part of the vicinity of the target parking position S of the all-around overhead image generated around the own vehicle position.

  As described above, the all-around bird's-eye view image generated by the viewpoint conversion unit 11 converts the images captured by the four in-vehicle cameras 1 to 4 into bird's-eye images each having a road surface as a projection plane, and these four bird's-eye images. Therefore, the images of the parked vehicles VP1 and VP2, which are three-dimensional objects having a height in this all-around bird's-eye view image, are collapsed in a direction away from the camera that captures these parked vehicles VP1 and VP2. It becomes. For this reason, when the target parking position S is on the extension line in the imaging direction of the parked vehicles VP1 and VP2 by the camera, the image of the parked vehicles VP1 and VP2 falls on the target parking position S side on the all-around overhead image. Become.

  Here, when the host vehicle V passes the position A, the parked vehicle VP1 is captured by the front camera 1 (or the left side camera 4), and the parked vehicle VP2 is captured by the rear camera 2 (or the left side camera 4). However, since the target parking position S does not exist on the extension line in the imaging direction of the parked vehicles VP1 and VP2 by these cameras, as shown in FIG. 6A, the all-around overhead image generated by the viewpoint conversion unit 11 Above, the fall of the image of parked vehicles VP1 and VP2 to the target parking position S side does not occur.

  On the other hand, when the host vehicle V passes the position B, the parked vehicle VP1 is imaged by the left side camera 4, and the target parking position S exists on an extension line in the imaging direction of the parked vehicle VP1 by the left side camera 4. Therefore, as shown in FIG. 6B, the image of the parked vehicle VP1 falls to the target parking position S side. When the host vehicle V reaches the position C, the parked vehicle VP1 is captured by the rear camera 2, and the target parking position S exists on an extension line in the imaging direction of the parked vehicle VP1 by the rear camera 2. As shown in FIG. 6C, the image of the parked vehicle VP1 falls to the target parking position S side.

  Therefore, in the parking scene shown in FIG. 5, when the entire surroundings overhead image generated by the viewpoint conversion unit 11 is displayed on the display 5 as it is, when the host vehicle V passes the position B or reaches the position C, etc. Depending on the position of the vehicle V, the target parking position S is blocked by the fall of the image of the parked vehicle VP1, and the driver of the vehicle V recognizes the target parking position S on the all-around overhead image displayed on the display 5. It becomes difficult. Therefore, in the parking assist device of the present embodiment, the display control unit 16 of the image processing device 10 is based on the relationship between the position of the host vehicle V, the target parking position S, and the position of the three-dimensional object (for example, the parked vehicle VP1). Then, it is determined whether or not the three-dimensional object (for example, the parked vehicle VP1) detected by the three-dimensional object detection unit 14 on the all-around overhead image generated by the viewpoint conversion unit 11 falls to the target parking position S side. When it is determined that the three-dimensional object (for example, the parked vehicle VP1) falls to the target parking position S side, the image of the three-dimensional object (for example, the parked vehicle VP1) is compressed to a size that does not overlap the target parking position S. By displaying on the display 5, even if there is a three-dimensional object in the vicinity of the target parking position, it is possible to display an all-around bird's-eye view image that can always accurately recognize the target parking position regardless of the position of the host vehicle. I am doing so.

  FIG. 7 shows that when the host vehicle V passes the position B in the parking scene shown in FIG. 5, the display control unit 16 of the image processing device 10 compresses the image of the parked vehicle VP <b> 1 that is a three-dimensional object and displays the display 5. It is a figure explaining a specific example of the method of displaying on. Here, compression refers to conversion of image data into other data in which the data amount is reduced (data is thinned out) while maintaining substantial properties.

  As shown in FIG. 7A, on the all-around bird's-eye view image, the direction along the image edge on the side away from the target parking position S of the parked vehicle VP1 is defined as the X axis, and the direction orthogonal thereto is defined as the Y axis. And The display control unit 16 of the image processing apparatus 10 determines the length D along the Y-axis direction of the image of the parked vehicle VP1 and the target parking position S of the parked vehicle VP1 at each point in the X-axis direction of this all-around overhead image. The distance D ′ from the image edge on the side away from the target parking position S to the target parking position S is measured. Then, the compression rate α of the image of the parked vehicle VP1 is calculated by α = D ′ / D, and the image of the parked vehicle VP1 is compressed at this compression rate α. As a result, the image of the parked vehicle VP1 is compressed to a size that does not overlap the target parking position S and displayed on the display 5 as shown in FIG.

  The compression rate α is calculated at each point in the X-axis direction, but the image of the parked vehicle VP1 may be compressed using the maximum value among them, or each image in the X-axis direction may be compressed. You may make it compress the image of parked vehicle VP1 by the compression rate which differs in a point. In addition, the compression of the image of the parked vehicle VP1 is based on the premise that the image of the parked vehicle VP1 is compressed to a size that does not overlap the parking target position S. However, the parking target position S is recognized by white line recognition processing for the entire surrounding image. In such a case, the image of the parked vehicle VP1 may be compressed to a size that does not overlap the white line including the white line that is a guide for the target parking position S.

  As described above, when the image of the parked vehicle VP1 is compressed to a size that does not overlap with the target parking position S, the image of the parked vehicle VP1 has fallen in the target parking position S on the all-around overhead image. In this case, an image is lost. Therefore, the display control unit 16 compensates for the image missing portion on the all-around bird's-eye view image generated by the image compression using the image of the past target parking position S so as to alleviate the uncomfortable feeling as an image. Yes. That is, the image processing device 10 is provided with an image storage unit 15, and an image of the target parking position when the image of the three-dimensional object has not fallen to the target parking position side is stored in the image storage unit 15. . The display control unit 16 uses the image of the target parking position stored in the image storage unit 15 to complement an image missing portion on the all-around bird's-eye image generated by image compression, and makes the all-around bird's-eye view image uncomfortable. It is made to display on the display 5 after adjusting to a non-image.

  In the above description, the size of the image of the parked vehicle VP1 on the all-around bird's-eye view image is measured to obtain the compression rate α of the parked vehicle VP1 image. Then, as described above, since the predicted trajectory until the host vehicle reaches the target parking position is calculated by the predicted trajectory calculation unit 13 of the image processing apparatus 10, the image of the three-dimensional object at each position on the predicted trajectory. It is possible to estimate in advance the degree of falling toward the target parking position. Therefore, if the compression rate of the three-dimensional object image at each position on the predicted trajectory is obtained in advance using this estimation result, the compression rate of the three-dimensional object image is appropriately adjusted according to the movement of the vehicle. It is possible to easily perform the optimal image compression while changing the mode.

  FIG. 8 shows the movement trajectory of the own vehicle V shown in FIG. 5 as the predicted trajectory of the own vehicle, that is, the vehicle arrives at the turn-back position C via the positions A and B in FIG. When the movement trajectory that moves to the target parking position S while traveling backward is calculated, the degree of collapse of the image of the parked vehicle VP1 at each position on the predicted route is estimated in advance, and the compression rate of the image of the parked vehicle VP1 It is a figure explaining the method to change this according to the movement of the own vehicle V. FIG.

  First, as shown in FIG. 8A, the display control unit 16 of the image processing apparatus 10 has an all-around bird's-eye view image (in this case, the own vehicle V is not collapsed) on the target parking position S side. The feature points p1 to p4 at the four corners of the side surface on the target parking position S side of the image of the parked vehicle VP1 are detected using the all-around bird's-eye view image when passing the position A shown in FIG. The feature points p1 to p4 of the image of the parked vehicle VP1 can be detected, for example, by performing known edge detection processing, vehicle shape estimation, or the like on the all-around overhead image. Of these feature points p1 to p4, the feature point p2 and the feature point p3 are contact points with respect to the road surface when the shape of the parked vehicle VP1 is approximated by a rectangular parallelepiped shape and regarded as an object on the road surface. The feature point p2 and the feature point p3 may be detected as follows, for example. That is, the front and rear ends of the vehicle floor surface specified by edge detection are defined as a feature point p2 ′ and a feature point p3 ′, and a point where a straight line passing through the feature point p1 and the feature point p2 ′ intersects the road surface is a feature point p2 and a feature point p4. By setting the point where the straight line passing through the feature point p3 ′ intersects the road surface as the feature point p3, the feature point p2 and the feature point p3 which are the ground contact points on the road surface can be detected.

  As described above, the feature point p2 and the feature point p3 among the feature points p1 to p4 of the image of the parked vehicle VP1 are the ground contact points on the road surface, so even if the position of the host vehicle V changes, these feature points. The positions of p2 and feature point p3 on the all-around overhead image are not changed. On the other hand, since the feature point p1 and the feature point p4 are upper end points in the height direction of the parked vehicle VP1, the positions of the feature point p1 and the feature point p4 on the all-around bird's-eye view image change according to the position of the own vehicle V. However, the positions of the feature point p1 and the feature point p4 on the all-around bird's-eye view image can be estimated from the positional relationship between the feature point p2, the feature point p3, and the own vehicle V. Then, if the positions of the feature point p1 and the feature point p4 of the image of the parked vehicle VP1 are known, the length D along the Y-axis direction of the image of the parked vehicle VP1 can be obtained as shown in FIG. The compression rate α (α = D ′ / D) for compressing the image of the parked vehicle VP1 to a size that does not overlap the target parking position S is obtained in advance for each position on the predicted trajectory of the host vehicle V. be able to.

  Therefore, when the movement trajectory of the own vehicle V shown in FIG. 5 is calculated as the predicted trajectory of the own vehicle, for example, the compression rate α1 of the image of the parked vehicle VP1 when the own vehicle V passes the position B, The compression rate α2 of the image of the parked vehicle VP1 when the vehicle V reaches the position C is obtained in advance, and the compression rate α1 and the compression rate are determined based on the movement amount of the own vehicle V between the position B and the position C. If the change of the rate α2 is linearly converted as shown in FIG. 8C and the relationship between the movement amount of the own vehicle V and the change of the compression rate α of the image of the parked vehicle VP1 is obtained, Optimal image compression can be easily performed while appropriately changing the compression ratio of the image of the parked vehicle VP1 in accordance with the movement. In addition, if the image of the parked vehicle VP1 is compressed at a compression rate that linearly changes in accordance with the amount of movement of the host vehicle V, the parked vehicle VP1 on the all-around overhead image displayed on the display 5 is displayed. The change of the image can be smoothed. Although an example in which the compression rate α of the image of the parked vehicle VP1 (three-dimensional object) is linearly changed according to the amount of movement of the own vehicle is described here, of course, it is nonlinear with respect to the amount of movement of the own vehicle. The changing compression rate α may be obtained, and the image of the three-dimensional object may be compressed at a compression rate that changes nonlinearly in accordance with the movement of the host vehicle.

  In addition, since the image of the three-dimensional object having a height on the all-around overhead image falls down in a direction away from the camera that images the three-dimensional object as described above, the target of the three-dimensional object image on the predicted route of the own vehicle The position where the falling to the parking position side starts can be specified from the positional relationship among the target parking position, the position of the three-dimensional object, and the position of the camera that images the three-dimensional object. For example, when the movement trajectory of the host vehicle V shown in FIG. 5 is calculated as the predicted trajectory, as shown in FIG. 9A, a part of the outer peripheral edge of the target parking position S on the parked vehicle VP1 side, When a part of the outer periphery on the target parking position S side of the parked vehicle VP1 and the position of the camera (here, the left side camera 4) mounted on the host vehicle V look down in a direction perpendicular to the road surface. The position on the predicted trajectory lined up on the straight line can be specified as the fall start position where the fall of the image of the parked vehicle VP1 that is a three-dimensional object starts toward the target parking position S.

  In addition, the amount of falling of the three-dimensional object toward the target parking position is determined by the position where the own vehicle is farthest from the three-dimensional object and the target parking position on the predicted route (position C on the movement locus of the own vehicle V shown in FIG. 5). Therefore, the maximum fall position where the three-dimensional object falls to the target parking position side on the predicted trajectory of the vehicle and the fall amount at that position must be specified in advance. Can do.

  Therefore, the fall start position, the maximum fall position, and the fall amount on the predicted route of the own vehicle are specified in advance, and linearly between the fall start position and the fall maximum position as shown in FIG. 9B. If the compression ratio of the image of the three-dimensional object that changes in advance is obtained in advance, the three-dimensional object image is appropriately adjusted with a linearly changing compression ratio according to the movement of the vehicle between the start-up position and the maximum position of the inset. The image of the three-dimensional object on the all-around overhead image displayed on the display 5 can be made smoother.

  By the way, the all-around bird's-eye view image generated by the viewpoint conversion unit 11 is an image obtained by connecting the four bird's-eye view images A1 to A4 obtained by viewpoint-converting the captured images of the four in-vehicle cameras 1 to 4 as described above. For this reason, the in-vehicle camera that captures a three-dimensional object existing in the vicinity of the target parking position may be switched as the host vehicle moves. For example, when the parked vehicle VP1 that is a three-dimensional object exists in the vicinity of the target parking space S, the position of the parked vehicle VP1 is the position of the left side camera 4 when the own vehicle passes the position shown in FIG. The image of the parked vehicle VP1 on the all-around bird's-eye view image is an image obtained by subjecting the captured image of the left side camera 4 to viewpoint conversion. On the other hand, when the own vehicle moves to the position shown in FIG. 10B, the position of the parked vehicle VP1 becomes the imaging range of the rear camera 2, and the image of the parked vehicle VP1 on the all-around overhead image is the captured image of the rear camera 2. Is obtained by converting the viewpoint.

  In this way, when the in-vehicle camera that captures a three-dimensional object in the vicinity of the target parking position is switched with the movement of the host vehicle, the degree of collapse of the three-dimensional object image on the all-around overhead image to the target parking position side is It changes for every vehicle-mounted camera which images a three-dimensional object. Therefore, when it is assumed that the vehicle-mounted camera that captures a three-dimensional object in the vicinity of the target parking position in the process of moving along the predicted trajectory, the vehicle's vehicle as shown in FIG. It is desirable to obtain the relationship between the amount of movement and the compression rate of the three-dimensional object image for each on-vehicle camera that captures the three-dimensional object image.

  For example, as in the example shown in FIG. 10, an in-vehicle camera that images a parked vehicle VP1 that is a three-dimensional object that exists in the vicinity of the target parking space S is moved along the predicted trajectory of the own vehicle. When switching from 4 to the rear camera 2, as shown in FIG. 11, on the predicted trajectory of the host vehicle, the left side camera 4 captures the parked vehicle VP <b> 1 and the rear camera 2 captures the parked vehicle VP <b> 1. Identify the section. Then, in the section in which the left vehicle 4 images the parked vehicle VP1, the fall start position when the parked vehicle VP1 is imaged by the left side camera 4 is specified by the method described above, and the parked vehicle VP1 The boundary position that deviates from the imaging range is set as the maximum collapse position, and the compression rate of the image of the three-dimensional object that linearly changes between the collapse start position and the maximum collapse position is obtained. Further, in the section in which the rear camera 2 images the parked vehicle VP1, the fall start position and the maximum fall position when the parked vehicle VP1 is imaged by the rear camera 2 are specified by the above-described method, and from the fall start position. The compression rate of the image of the three-dimensional object that linearly changes until the maximum collapse position is obtained.

  In this way, when it is assumed that the in-vehicle camera that captures a three-dimensional object in the vicinity of the target parking position in the process of moving along the predicted trajectory, as shown in FIG. By determining the relationship between the amount of movement and the compression rate of the image of the three-dimensional object for each in-vehicle camera that images the three-dimensional object, the image of the three-dimensional object is displayed even if the in-vehicle camera that images the three-dimensional object is switched. The three-dimensional object image can be appropriately compressed in accordance with the amount of movement, and an easy-to-see all-around bird's-eye view image that does not fall into the target parking position side can be displayed on the display 5.

  As described above in detail with reference to specific examples, according to the parking assistance device of the present embodiment, the display control unit 16 performs a three-dimensional object detection unit on the all-around overhead image generated by the viewpoint conversion unit 11. 14 determines whether the image of the three-dimensional object detected by 14 falls down to the target parking position side, and if it falls down, compresses the image of the three-dimensional object to a size that does not overlap the target parking position and displays it on the display 5. Therefore, even when there is a three-dimensional object in the vicinity of the target parking position, an image that can always accurately recognize the target parking position can be displayed on the display 5 regardless of the position of the own vehicle. Parking operation can be supported appropriately.

  In addition, according to the parking assistance device of the present embodiment, the display control unit 16 has the image storage unit 15 stored the image missing portion that is generated by compressing the three-dimensional object image on the all-around overhead image. Since it complements with the past image, the all-around overhead image without a sense of incongruity can be displayed on the display 5.

  Moreover, according to the parking assistance device of the present embodiment, the display control unit 16 estimates in advance the degree of collapse of the three-dimensional object image at each position on the predicted trajectory until the host vehicle moves to the target parking position, Based on the estimation result, by changing the compression rate of the image of the three-dimensional object in accordance with the movement of the host vehicle, the image of the three-dimensional object can be optimally and easily compressed and an easy-to-view image can be displayed on the display 5. .

  Moreover, according to the parking assistance apparatus of this embodiment, the display control part 16 is the fall start position where the fall to the target parking position side of the image of the three-dimensional object starts on the predicted route of the host vehicle, and the fall amount is maximum. And the compression rate of the image of the three-dimensional object is linearly changed with the movement of the host vehicle between the collapse start position and the maximum collapse position. The change in the image of the three-dimensional object on the surrounding overhead image can be made smoother.

  Moreover, according to the parking assistance apparatus of this embodiment, when the vehicle-mounted camera which images the image of a solid object switches with the movement of the own vehicle, the display control part 16 is the image of the solid object on an all-around bird's-eye view image. By changing the compression ratio of each vehicle-mounted camera that captures an image of the three-dimensional object, even if the vehicle-mounted camera that captures the three-dimensional object is switched, the image of the three-dimensional object is appropriately matched to the amount of movement of the host vehicle. Can be compressed.

  The embodiment described above is an example of application of the present invention, and the technical scope of the present invention is not intended to be limited to the contents described in the above embodiment. That is, the technical scope of the present invention is not limited to the specific technical matters disclosed in the above embodiments, but includes various modifications, changes, alternative techniques, and the like that can be easily derived from this disclosure.

  For example, in the embodiment described above, parking support having a function of causing the display 5 to display an all-around bird's-eye view image generated by connecting the bird's-eye view images obtained by viewpoint-converting the captured images of the four in-vehicle cameras 1 to 4. Although the present invention is an example applied to an apparatus, the present invention captures an image in a predetermined direction around a vehicle with a single in-vehicle camera, and generates and displays an overhead image by converting the viewpoint of the image. Any parking assistance device having a function of converting a viewpoint of a captured image of a camera into a bird's-eye view and displaying the image can be applied.

It is a block diagram which shows schematic structure of the parking assistance apparatus to which this invention is applied. It is a schematic diagram explaining the installation position of four vehicle-mounted cameras. It is a block diagram which shows the functional structure inside an image processing apparatus. It is a figure which shows the outline | summary of the all-around bird's-eye view image produced | generated by the viewpoint conversion part of an image processing apparatus. It is a schematic diagram which shows an example of a parking scene. FIG. 6 is a diagram illustrating an image example of an all-around overhead image generated by a viewpoint conversion unit of the image processing device in the parking scene illustrated in FIG. 5. It is a figure explaining a specific example of the method of compressing and displaying the picture of a solid thing on a display. It is a figure explaining an example of the method of estimating the fall degree of the image of the solid object in each position on a prediction path | route in advance, and changing the compression rate of the image of a solid object according to the movement of the own vehicle. It is a figure explaining the other example of the method of estimating the fall degree of the image of the solid object in each position on a prediction path | route beforehand, and changing the compression rate of the image of a solid object according to the movement of the own vehicle. It is a figure explaining the example in which the vehicle-mounted camera which images the solid object of the vicinity of a target parking position switches with the movement of the own vehicle. It is a figure showing the relationship between the amount of movement of the own vehicle and the compression rate of the three-dimensional object near the target parking position, and when the in-vehicle camera that captures the image of the three-dimensional object switches with the movement of the own vehicle, It is a figure which shows the example which calculated | required the compression rate of the image for every vehicle-mounted camera.

Explanation of symbols

1 Front camera (vehicle camera)
2 Rear camera (vehicle camera)
3 Right side camera (vehicle camera)
4 Left side camera (vehicle camera)
DESCRIPTION OF SYMBOLS 5 Display 6 Operation input device 10 Image processing apparatus 11 View point conversion part 12 Target parking position recognition part 13 Predicted locus calculation part 14 Solid object detection part 15 Image storage part 16 Display control part

Claims (9)

An imaging means mounted on the host vehicle for capturing an image around the host vehicle;
Viewpoint conversion means for generating a bird's-eye view image by performing viewpoint conversion processing on the image captured by the imaging means;
Display means for displaying an overhead image generated by the viewpoint conversion means;
On the overhead image generated by the viewpoint conversion means, target parking position recognition means for recognizing a target parking position that is a position for parking the own vehicle;
A three-dimensional object detection means for detecting a three-dimensional object in the vicinity of the target parking position;
Whether or not the image of the three-dimensional object falls to the target parking position side on the overhead image generated by the viewpoint conversion unit based on the relationship between the position of the host vehicle, the target parking position, and the position of the three-dimensional object A fall-down judging means for judging
And an image adjusting unit that compresses the three-dimensional object image on the overhead image when the three-dimensional object image is determined to fall to the target parking position side by the falling-down determining unit. Support device. A predictive trajectory calculating means for calculating a predictive trajectory until the host vehicle reaches the target parking position;
The fall determination means estimates a fall degree of the image of the three-dimensional object at the target parking position side at each position on the prediction locus calculated by the prediction locus calculation means,
The parking according to claim 1, wherein the image adjustment unit changes a compression rate of the image of the three-dimensional object in accordance with the movement of the own vehicle based on the estimation result of the degree of collapse by the collapse determination unit. Support device. The fall determination means includes a fall start position at which a fall of the three-dimensional object image toward the target parking position is started on the prediction locus calculated by the prediction locus calculation means, and the target parking position of the three-dimensional object image. Specify the maximum position where the maximum amount of falling to the side
The image adjustment means linearly changes a compression rate of the image of the three-dimensional object as the vehicle moves between a fall start position specified by the fall determination means and a maximum fall position. Item 3. The parking assistance device according to item 2. A storage means for storing an image of the target parking position when the image of the three-dimensional object has not collapsed toward the target parking position;
4. The image adjustment unit complements an image missing portion on the overhead image generated by compressing the image of the three-dimensional object with an image stored in the storage unit. 5. The parking assistance device according to one item.   The parking assist device according to any one of claims 1 to 4, wherein the target parking position recognizing unit recognizes the target parking position by white line recognition processing for the overhead view image generated by the viewpoint conversion unit. .   6. The three-dimensional object detection unit according to claim 1, wherein the three-dimensional object detection unit detects a three-dimensional object existing in the vicinity of the target parking position by performing pattern recognition processing on the overhead image generated by the viewpoint conversion unit. The parking assistance device according to any one of the above. The imaging means comprises a plurality of imaging means for capturing images of different areas around the vehicle,
The viewpoint conversion unit performs a viewpoint conversion process on the images captured by the plurality of imaging units to generate a plurality of overhead images, and combines the plurality of overhead images to generate a combined overhead image.
The parking support apparatus according to any one of claims 1 to 6, wherein the display unit displays a composite overhead image generated by the viewpoint conversion unit.   The image adjustment unit is configured to display the three-dimensional object on the composite overhead image generated by the viewpoint conversion unit when an imaging unit that captures an image of the three-dimensional object detected by the three-dimensional object detection unit is switched as the vehicle moves. The parking support apparatus according to claim 7, wherein the image compression rate is changed for each imaging unit that captures an image of the three-dimensional object. An image display method for converting an image around a host vehicle imaged by an imaging unit mounted on the host vehicle into a bird's-eye view image and displaying the image on a display unit,
While recognizing the target parking position, which is the position where the vehicle is parked on the overhead view image, detecting a three-dimensional object in the vicinity of the target parking position,
Based on the relationship between the position of the host vehicle and the target parking position and the position of the three-dimensional object, it is determined whether the image of the three-dimensional object falls on the target parking position side on the overhead image,
An image display method comprising compressing a solid object image on the overhead image and displaying the compressed image on the display unit when it is determined that the solid object image falls to the target parking position side.

Images