JP2009049943A - Top view display unit using range image - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a top view display unit using a range image capable of readily recognizing a three-dimensional object that is an impediment to a vehicle in displaying a top view image. <P>SOLUTION: A camera image capture unit 1 captures images photographed outside the vehicle by a plurality of cameras. A top view image formation unit 2 uses the captured images to compose images from a virtual viewpoint above the vehicle around the vehicle. A range view processing unit 3 uses the images captured by the camera image capture unit 1 to detect the height of the three-dimensional object, and uses information on heights of the three-dimensional objects to display the three-dimensional objects having heights equal to or greater than a given height that are an impediments to the vehicle in a different manner from that of displaying other image portions, that is, in such a manner that the brightness, saturation, or chromaticity thereof gradually changes according to the heights of the objects. Alternatively, the three-dimensional objects having heights equal to or greater than a given height are displayed by being filled with the same colors or displayed in white colors. Alternatively, a three-dimensional object that is highest among those displayed on the screen may be displayed in a different manner from that of displaying others, and curbstones or the like may also be displayed. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a top view display device that displays an image of the surroundings of a vehicle having a virtual viewpoint on the upper part of the vehicle by photographing the outside with a camera provided in the vehicle, and particularly displays a high obstacle in an easy-to-understand manner using a distance image. The present invention relates to a top view display device using a distance image that can identify an obstacle.

  When putting a vehicle in a narrow garage or parking lot, the driver needs to be very careful and sometimes accidents such as rubbing the surrounding vehicle with the surrounding object may occur. Also, when taking a vehicle out of a garage or parking lot, it is necessary to pay close attention to surrounding objects and people. In addition to entering and exiting garages and parking lots, vehicles such as driving on extremely narrow roads or contacting with surrounding objects when passing on opposite roads on narrow roads, or preventing wheels from falling into grooves, etc. There are many cases where sufficient attention is required around the area.

  As countermeasures, the rear of the vehicle, which has been widely installed in vehicles in recent years, is photographed and displayed on a monitor, and a rear-facing camera that facilitates parking and parking, When a vehicle leaves a narrow road with poor visibility, use a camera outside the vehicle, such as a side-view camera, to check the vehicle traveling on the other road, and monitor the situation around the vehicle. Driving is also done.

  However, even if an ultra-wide-angle camera is used as such a camera outside the vehicle, for example, when a vehicle is put in the garage while the vehicle is moved backwards, and there are obstacles around the rear side of the vehicle, the vehicle is used for backward shooting when the vehicle is moved backward. Being out of the camera's field of view, it becomes impossible to know the state of the obstacle. As a countermeasure, it is also conceivable to provide super wide-angle cameras on both sides of the vehicle so that the portion outside the field of view of the rear photographing camera can be seen by the side photographing camera.

  However, in the image of the ultra-wide-angle camera as described above, the appearance of the object is greatly different between the central part and the peripheral part of the photographed image, and the positional relationship between the object and the vehicle is clear just by looking at the photographed image. Instead, it may cause unexpected collisions and contact accidents. In particular, even if the image from the rear camera is switched to the image from the side camera, the image of the object such as an obstacle displayed on the screen at that time is very different, and the actual state of the object must be grasped. Is not easy.

  Therefore, a plurality of cameras for shooting outside the vehicle are mounted on the vehicle, the images shot by these cameras are combined, and an image is formed as if the surroundings of the vehicle were shot from the camera installed above the vehicle. It has been proposed to be displayed. When using such a plurality of cameras, for example, as shown in FIG. 7, a vehicle right side shooting camera 12 for shooting the right side of the vehicle 11, a vehicle left side shooting camera 13 for shooting the left side of the vehicle 11, A total of four cameras, a rear camera 14 for photographing the rear and a front camera 15 for photographing the front side of the vehicle 11, are mounted, and all the images of these cameras are synthesized. For example, FIG. As schematically shown in c), it has been proposed to display on the monitor an image as if the entire periphery of the vehicle 11 was photographed by a single camera provided directly above the vehicle. This method is also referred to as a top view system, and two or three cameras for photographing the outside of the vehicle are appropriately selected as necessary, and the installation positions are also selected and installed at various positions.

In addition, a normal camera and an infrared camera that shoot in the same direction are used as surveillance cameras outside the vehicle, and the captured image of the normal camera is displayed on the monitor, and the dangerous object is monitored by the infrared camera. Japanese Patent Application Laid-Open No. 2004-228688 discloses a technique for easily displaying a dangerous object by increasing the brightness of a video portion at a position where the object exists in a normal camera image displayed on a monitor. Further, Patent Document 2 discloses a technique for displaying a stereoscopic image by displaying a two-dimensional image on each of a plurality of display surfaces having different depth positions and changing the luminance of each two-dimensional image.
JP 2005-184523 A JP 2000-214413 A

  When the image of the vehicle surroundings is displayed with the top view as described above, it is generally possible to display the surrounding situation in an easy-to-see manner. However, regarding the display of the three-dimensional object, the three-dimensional object is displayed in a collapsed state. It was difficult to intuitively recognize the distinction of a flat object. Also, for example, when there is a curb around the road or parking lot, and the height is high from that part, it is difficult to easily grasp the state from the screen because the surrounding objects are the same color.

  Therefore, the present invention recognizes a three-dimensional object around the vehicle easily and reliably when a top view image having a virtual viewpoint above the vehicle is displayed by synthesizing images taken by a camera that captures the outside of the vehicle. It is a main object of the present invention to provide a top view display device using a distance image that can be used.

  In order to solve the above-described problem, a top view display device using a distance image according to the present invention has a camera image capturing unit that captures a captured image of a camera that captures the outside of the vehicle, and an image of the camera image capturing unit. A top view image forming unit that synthesizes an image around the vehicle with a virtual viewpoint, a distance image processing unit that detects a three-dimensional object based on an image of the camera image capturing unit, and a three-dimensional image obtained by the distance image processing unit A top-view three-dimensional object display processing unit for displaying a three-dimensional object having a predetermined height or more in a different manner from the others based on the object height information is provided.

  The top-view display device using another distance image according to the present invention may be configured such that, in the top-view display device, a three-dimensional object having a predetermined height or more has a higher brightness, saturation, or chromaticity. The display is characterized by being gradually changed. Further, the three-dimensional object having the predetermined height or higher is displayed by being filled with the same color, and the three-dimensional object having the predetermined height or higher is displayed in white, and the predetermined object is further displayed. The three-dimensional object display having a height equal to or higher than the height is characterized in that the highest three-dimensional object in the screen display is displayed in a mode different from the others.

  Since the present invention is configured as described above, it becomes an obstacle for a vehicle when displaying a top view image having a virtual viewpoint above the vehicle by combining images captured by a plurality of cameras that capture the outside of the vehicle. It becomes possible to easily recognize a three-dimensional object around the vehicle.

  The present invention provides a camera image capturing unit that captures a captured image of a camera that captures the outside of a vehicle for the purpose of easily recognizing a three-dimensional object that becomes an obstacle for a vehicle when displaying a top view image. A top-view image forming unit that synthesizes an image around the vehicle with a virtual viewpoint above the vehicle from the image of the camera image capturing unit, and a distance image process that detects a three-dimensional object from the image of the camera image capturing unit This is realized by displaying a three-dimensional object having a predetermined height or higher in a different manner from the other, based on the height information of the three-dimensional object obtained by the unit and the distance image processing unit.

  FIG. 1 shows a functional block diagram of an embodiment of the present invention. In the embodiment shown in FIG. 1, as shown in FIG. 7, a camera image capturing unit 1 that captures captured images of a plurality of cameras that capture the outside of the vehicle is provided. The image is input to the top view image forming unit 2 that synthesizes an image with a virtual viewpoint above, and among the plurality of camera images captured by the camera image capturing unit 1, for example, images of a plurality of cameras such as left and right cameras are selected. The distance image processing unit 3 that forms a stereoscopic image is also input.

  In the distance image processing unit 3, for example, a distance image can be obtained by a stereo method using a conventionally known epipolar geometry as shown in FIG. That is, in the distance image processing by the stereo method, when the gazing point P in the three-dimensional space is projected as two spherical images P1 and P2, each point exists on the epipolar line, and the three-dimensional space The gazing point P, the projected points P1 and P2, and the optical center of the camera exist on one plane, and this plane is called an epipolar plane. This geometric light beam is called epipolar geometry and satisfies the epipolar equation. A distance image can be obtained by giving an actual measurement value to the equation and solving it.

  As the three-dimensional object image processing for obtaining the distance image, in addition to the stereo image method using the epipolar geometry as described above, various three-dimensional object image processing such as the illuminance difference stereo method and the Helmholtz stereo method conventionally used are used. Techniques can be used. In this case, in addition to knowing the existence of a three-dimensional object from images of a plurality of cameras, it is also possible to know the existence of a three-dimensional object by analyzing an image taken by the same camera by moving a vehicle with one camera. The distance image processing for knowing the height of the three-dimensional object can also be performed using these methods.

  The three-dimensional object image display processing unit 4 uses various methods such as changing the brightness according to the height, among various methods as will be described later, according to the image data subjected to the image processing by the stereo method. A solid object is displayed. In the top-view image three-dimensional object display processing unit 5, the three-dimensional object processed by the three-dimensional object image display processing unit 4 is displayed in the top view image formed by the top view image forming unit 2 as described above. Perform the process. As a result, the presence of a three-dimensional object can be displayed in an easy-to-view manner by various methods with respect to an image in which only a top view image has been conventionally formed.

  The three-dimensional object display based on the distance image obtained as described above employs a method of changing the luminance depending on the height, for example, as shown in FIG. In this figure, when a vehicle is put into a garage surrounded by walls, the plane including the ground is darkest, and the height of the solid object becomes higher as the height of the solid object is 20cm, 40cm, 60cm, 80cm. indicate. Thereby, the difference between a plane and a three-dimensional object can be clearly distinguished, and the height can be easily grasped. Contrary to the above, the display different depending on the height is displayed by a color difference such that the higher the three-dimensional object is, the darker the color is displayed, or the higher the three-dimensional object is, the image changes from blue to red. It can be displayed by various methods. That is, as the three-dimensional object is higher, the display is performed by gradually changing any one of luminance, saturation, and chromaticity.

  In addition, for example, as shown in FIG. 5A, the three-dimensional object portion may be filled with the same color. That is, in the case of the illustrated garage, the plane G is a normal screen, and the detected wall W as a three-dimensional object is covered with a light yellow mask, for example, and the three-dimensional object is simply painted light yellow on the screen. Do. By painting a solid object in this way, it shows that there is a solid object there, does not indicate what it is, and sees the image by recognizing what the three-dimensional object is. It is possible to prevent the occurrence of an accident due to a psychology that collides with an object or becomes too conscious by recognizing a three-dimensional object and instead approaches the three-dimensional object.

  Further, as shown in FIG. 5B, the ground G may be a normal photographed image, and the walls W as solid objects may be all white so as to distinguish between a planar object and a solid object. Even if the three-dimensional object is all white as described above, it is possible not to indicate what the three-dimensional object is, and to give the driver a misunderstanding of the three-dimensional object or not to give excessive consciousness. .

  Further, for example, as shown in FIG. 6 (a), for a car stop S having a height of about 15 cm in an outdoor parking lot or the like, the highest part of the entire vehicle periphery is calculated, and the part is displayed darkly or displayed in red. By highlighting this, for example, the car stop S that is originally the same color as the surroundings and is difficult to see can be seen very clearly on the screen. In that case, first, a particularly important obstacle higher than 20 cm, for example, is detected in the display screen, and when there is no such object, the highest one is detected in the screen as described above. When it is 20 cm, it may be highlighted as described above.

  Also, as shown in the figure (b), when there is a curbstone at the rear of the car, this part, which is the highest in the surroundings, should be clearly indicated by a dark or red display etc. Can do. Thus, by selecting and showing the highest part in the display image, even a relatively low object of about 20 cm can be clearly shown as an object to be noted for the vehicle.

  In the present invention, which is composed of functional blocks as shown in FIG. 1 and displays various three-dimensional objects as described above, for example, it can be implemented by sequentially operating according to the operation flow shown in FIG. That is, in the three-dimensional object display process in the top view image shown in FIG. 2, images are first captured from a plurality of cameras (step S1), and then the camera images are combined to form a top view image (step S1). S2). This operation is performed by capturing an image with the camera image capturing unit 1 of FIG. 1 and forming a top view image with the top view image forming unit 2.

  Next, from a plurality of camera images, for example, a plurality of cameras such as left and right cameras are selected and stereo image processing is performed (step S3), and three-dimensional coordinates are acquired from the captured images (step S4). For the three-dimensional object including the height information obtained by the three-dimensional coordinates, for example, the three-dimensional object image display processing of various aspects as described above is performed (step S5). . Thereafter, the three-dimensional object image obtained in step S5 is combined with the top view image obtained in step S2, and the three-dimensional object is displayed in various modes as shown in FIGS.

It is a functional block diagram of the Example of this invention. It is an operation | movement flowchart of the Example. It is explanatory drawing of the stereo method which acquires a distance image from a camera image. It is a figure which shows the example which displays a solid object in a top view image. It is a figure which shows the other example which displays a solid object in a top view image. It is a figure which shows the further another example which displays a solid object in a top view image. It is explanatory drawing of the top view system proposed conventionally.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Camera image capture part 2 Top view image formation part 3 Distance image process part 4 Solid object image display process part 5 Solid object display process part in a top view image

Claims (5)

A camera image capturing unit that captures a captured image of a camera that captures the outside of the vehicle;
A top-view image forming unit that synthesizes an image around the vehicle with a virtual viewpoint above the vehicle, based on the image of the camera image capturing unit;
A distance image processing unit that detects a three-dimensional object from the image of the camera image capturing unit;
A solid object display processing unit in a top view image that displays a solid object of a predetermined height or higher in a different manner from the other, based on the height information of the solid object obtained by the distance image processing unit. A top view display device with a characteristic distance image.   The top-view display device using a distance image according to claim 1, wherein the three-dimensional object display processing unit in the top-view image displays by changing the luminance, saturation, or chromaticity gradually as the height increases. .   The top-view display device using a distance image according to claim 1, wherein the three-dimensional object display processing unit in the top-view image is displayed with the same color.   The top view display device using a distance image according to claim 1, wherein the three-dimensional object display processing unit in the top view image displays in white.   2. The top view display device according to claim 1, wherein the three-dimensional object display processing unit in the top view image displays the highest three-dimensional object in the screen display in a mode different from the others.

Images