JP2008048094A - Video display device for vehicle, and display method of video images in vicinity of the vehicle - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To detect an obstacle from a displayed bird's-eye view image and to make a driver easily recognize the obstacle, without causing enlarging or complicating of constitution or increasing the cost. <P>SOLUTION: An image for which the rear part of a vehicle 4 is imaged with a camera 1 at a time (t-Δt) is transformed two-dimensionally in a bird's-eye view transformation processing part 21, to prepare a first bird's-eye view image, the first bird's-eye view image is shifted, on the basis of the moving amount of the vehicle 4, and the difference between the shifted first bird's-eye view image and a second bird's-eye view image, prepared by two-dimensionally transforming the image imaged with the camera 1 at time t in the bird's-eye view transformation processing part 21 is extracted. The prepared differential bird's-eye view image is superimposed on the second bird's-eye view image and is composited in an image composite processing part 23, an obstacle 5 with a height is detected in the composited bird's-eye view image, and the composited bird's-eye view image is displayed on a display 3. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a vehicular video display device that displays a video image of the surroundings of a vehicle and a method for displaying a video around the vehicle in order to make a driver recognize the situation around the vehicle.

  2. Description of the Related Art Conventionally, a technique for converting an actual image into an image viewed from another viewpoint by performing image processing on the actual image captured from a certain viewpoint by a fixed camera is known. Such image processing is called viewpoint conversion processing, and is used, for example, for imaging a blind spot area around the vehicle with an in-vehicle camera and displaying the blind spot area as a bird's-eye view image looking down from above.

  Such viewpoint conversion processing creates an overhead image on the assumption that an actual image captured by an actual camera is captured from above by a virtual camera. For this reason, when there is an object having a height in the real image, this object is displayed in a state of being uncomfortable (in a collapsed state) in the overhead view image, and it is difficult to recognize the actual situation. is there.

As a technique for eliminating a sense of incongruity when an object having the above height is displayed as a bird's-eye view image, for example, in Patent Document 1, it is obtained by relative movement of an imaging object and a camera as the vehicle travels. A technique is disclosed in which moving image data is analyzed to acquire height information of an imaging target, and a predetermined calculation is performed based on the height information, thereby creating a bird's-eye view image with no sense of incongruity.
JP 2002-34035 A

  However, in the conventional technique described in Patent Document 1 described above, moving image data obtained by relative movement of the imaging object and the camera as the vehicle travels is analyzed, and the height information of the imaging object is obtained. Since the overhead image of the object to be imaged is created in consideration of the obtained height information, it is necessary to analyze and calculate the moving image data with a large amount of information. Becomes complicated and requires a lot of processing time. Therefore, in order to display such a bird's-eye view image with little delay as the vehicle travels, it is necessary to execute arithmetic processing at high speed, which increases the size and complexity of the configuration and the system cost. There was a risk of incurring an increase.

  Therefore, the present invention has been made in view of the above, and it is possible to accurately detect an obstacle from a displayed overhead image without causing an increase in size and complexity of the configuration and an increase in cost. It is intended to make it easy to recognize.

  In order to achieve the above object, the means for solving the problems of the present invention includes an imaging means for imaging the periphery of a vehicle, and an overhead image creation means for creating an overhead image by converting the viewpoint of the image captured by the imaging means. The first bird's-eye view image created by converting the image picked up by the image pickup means at the first time by the bird's-eye view image creation means is converted into the vehicle at the second time based on the amount of movement of the vehicle. The second bird's-eye view image created by converting the first bird's-eye view image moved according to the position and the image taken by the imager at the second time by the bird's-eye image creator. The difference overhead image is extracted to create a difference overhead image, and the synthesized overhead image is superimposed on the second overhead image, and the overhead image synthesized by the image synthesis means is displayed. And display means.

  According to the present invention, it is possible to accurately detect and display an obstacle having a height existing in the traveling direction of the vehicle without increasing the size, complexity, and cost of the device. Can easily recognize obstacles.

  DESCRIPTION OF THE PREFERRED EMBODIMENTS The best embodiment for carrying out the present invention will be described below with reference to the drawings.

  FIG. 1 is a diagram showing a configuration of a vehicle image display apparatus according to Embodiment 1 of the present invention. The vehicle image display apparatus of the first embodiment shown in FIG. 1 accurately detects an obstacle mounted on the vehicle and having a height behind the vehicle and displays the overhead view so that the driver can recognize the obstacle behind the vehicle. The present invention is applied to a device that assists the driving operation of the driver.

  In FIG. 1, the vehicular video display apparatus includes a camera 1, a processing unit 2, and a display 3.

  The camera 1 is attached above the trunk behind the vehicle, and captures an image at an angle of view (a depression angle) such that the vehicle periphery including the road surface behind the vehicle 4 is looked down obliquely from above as shown in FIG. 2 or FIG. The camera 1 gives the captured image data to the processing unit 2.

  The processing unit 2 is realized by, for example, a microcomputer provided with a CPU, a storage device, an input / output device, and the like. The processing unit 2 reads the image data obtained by the camera 1, the vehicle speed signal, the rudder angle signal indicating the driver's steering wheel operation, and the like based on the read signal and the control logic (program) stored in advance in advance. A processing operation for executing and displaying the image processing described in the above is controlled. In order to perform such processing operation, the processing unit 2 includes an overhead conversion processing unit 21, an image temporary storage unit 22, and an image composition processing unit 23.

  The overhead view conversion processing unit 21 creates overhead image data based on the display position on the image data captured by the camera 1. Specifically, the overhead view conversion processing unit 21 performs the coordinate conversion by sequentially copying the image data to the working memory for each pixel based on a predetermined conversion table stored in a memory or the like, thereby performing the overhead view image data. Create The conversion table is a two-dimensional table indicating which points on the overhead image data are the same as the points on the image data in order to convert the image data captured by the camera 1 into overhead image data. This is uniquely determined based on the optical characteristics of the camera lens of the camera 1 and the mounting position and mounting angle with respect to the vehicle. The overhead view conversion processing unit 21 gives the overhead image data obtained by the conversion to the image temporary storage unit 22.

  The temporary image storage unit 22 temporarily stores the overhead image data obtained by the overhead conversion processing unit 21 in association with the display coordinates, and gives the stored overhead image data to the image composition processing unit 23.

  The image compositing processing unit 23 obtains the overhead image data obtained by converting the image data captured by the camera 1 at the imaging time t as the vehicle moves, and the imaging before the imaging time t by a minute time Δt. The overhead image data obtained by converting the image data captured by the camera 1 at time (t−Δt) is superposed with the amount of movement of the vehicle 4 in the minute time Δt taken into account, and the difference between the two overhead image data is calculated. The extracted difference overhead image data is subjected to enhancement processing or the like, and the overhead image subjected to this enhancement processing or the like is superimposed and synthesized on the overhead image captured at the imaging time t.

  The image composition processing unit 23 calculates the movement amount of the vehicle 4 based on the vehicle speed signal when the overhead image data at the imaging time t and the overhead image data at the imaging time (t−Δt) are superimposed. The overhead image data at the imaging time (t−Δt) is moved as coordinate data based on the amount, and is superimposed on the overhead image data at the imaging time t.

  The image composition processing unit 23 performs, for example, a process of enclosing the outline of the overhead image as an image enhancement process, or displays the extracted part as a color video and displays the other part as a monochrome video, thereby preventing the obstacle and other parts from being displayed. It is possible to identify and recognize The image composition processing unit 23 gives the display 3 the overhead image data obtained by the composition.

  The display 3 displays a bird's-eye view image based on the bird's-eye view image data synthesized by the image composition processing unit 23, and allows the driver to recognize an obstacle existing behind the vehicle by visually recognizing the bird's-eye view image displayed. .

  In such a vehicular video display device, the procedure shown in the flowchart of FIG. In FIG. 4, in the state where the vehicle 4 is moving backward, first, as shown in FIG. 2, the rear of the vehicle is imaged at the imaging time t as shown in FIG. 2, and for example, an image as shown in FIG. Here, as shown in FIG. 2, it is assumed that an obstacle 5 having a height exists behind the vehicle 4, and the vehicle 4 moves backward and approaches the obstacle 5. Subsequently, the image data obtained by the imaging at the imaging time t is given to the overhead view conversion processing unit 21 of the processing unit 2, and is converted into the overhead image data by the overhead view conversion processing unit 21, for example as shown in FIG. An overhead image is obtained (step S42).

  After that, the image is captured at the imaging time (t−Δ1) which is a minute time Δt before the previous imaging time t, that is, the current imaging time, and is converted and stored in the temporary image storage unit 22 in the same manner as this time (imaging time t). The overhead image data captured last time is read from the image temporary storage unit 22 to the image composition processing unit 23 (step S43). Here, in the previous imaging, for example, an image as shown in FIG. 7 is obtained, and it is assumed that an overhead image as shown in FIG. Subsequently, the overhead image data obtained by the current overhead conversion process is stored and stored in the image temporary storage unit 22 (step S44).

  Next, the movement amount of the vehicle 4 between the current imaging time t and the previous imaging time (t−Δt), that is, the minute time Δt is calculated based on the vehicle speed (step S45). After the calculation, the previous time shown in FIG. 8 is obtained by moving the previous bird's-eye image data by the previously calculated amount of movement with reference to the road surface so that the previous bird's-eye image data matches the previous bird's-eye image data. The overhead image for imaging is, for example, an overhead image as shown in FIG. 9 (step S46).

  Thereafter, the current bird's-eye view image data and the moved previous bird's-eye view image data are superimposed. That is, the overhead image shown in FIG. 6 and the overhead image shown in FIG. 9 are superimposed. The superimposed bird's-eye view image data has the same road surface condition, but the obstacle 5 has a different depression angle with respect to the camera 1 due to the backward movement of the vehicle 4, so that the obstacle 5 is an image having a different size and shape from the previous time and this time. The bird's-eye view image of the obstacle 5 displayed is not consistent as shown in FIG. Therefore, a non-matching portion is extracted from the superimposed overhead image data as difference overhead image data (step S47). At this time, by shortening the imaging interval, that is, the minute time Δt, the difference overhead image data represents the contour of the obstacle 5 and the contour of the obstacle 5 can be extracted. Thereby, the obstacle 5 is detected accurately.

  Subsequently, image processing is performed to highlight the extracted difference overhead image data. For example, the extracted difference overhead image data is subjected to image processing so as to become a predetermined single color display (step S48). Finally, the overhead image data obtained by this imaging is monochromatic and combined with the difference overhead image data that expresses the outline of the obstacle 5, and the overhead image obtained by the synthesis is displayed on the display 3. (Step S49).

  The bird's-eye view image synthesized and displayed on the display 3 becomes a bird's-eye view image in which the outline of the obstacle 5 is highlighted by monochromatization, so that the bird's-eye view image effective for recognition of the obstacle 5 can be displayed. Thereby, the driver can recognize the obstacle 5 easily and accurately by visually recognizing this display, and can assist the driver in driving operation.

  In the apparatus of the above embodiment, since the overhead image is obtained by coordinate conversion of the two-dimensional data, it is faster than the conventional method of obtaining the overhead image by processing the three-dimensional moving image data. Therefore, the apparatus can be realized with a configuration that is relatively small, simple, and inexpensive as compared with the prior art.

  Note that the detected obstacle 5 may be highlighted. For example, the obstacle 5 may be displayed in a specific color set in advance, and other parts other than the obstacle 5 may be displayed in monochrome. Alternatively, the detected obstacle 5 may be highlighted by surrounding it with a specific frame. By doing in this way, it becomes possible to raise the recognition property of the obstacle 5 to a driver | operator.

  In the above description, the obstacle 5 is described as a stationary object. However, even when the obstacle 5 is a moving object, it is possible to display and detect an overhead image of the moving object in the same manner as described above. Similar effects can be obtained.

  Furthermore, in the said Example, although it imaged with the single camera, when displaying an overhead image similarly using a some camera and detecting an obstruction, it imaged with each camera. In addition to the method of detecting obstacles separately for each overhead image obtained by converting images, and then combining the overhead images to detect obstacles, images captured by multiple cameras It is also possible to adopt a method of detecting an obstacle in a state where the overhead images obtained by conversion are combined.

  In addition, when creating a bird's-eye view image with a plurality of cameras, the image is distorted along the optical axis of each camera in the overlapped portion of each image captured by the plurality of cameras, so the distance from the obstacle It is possible to measure and display this distance together.

  Specifically, for example, when a rear camera is installed in the trunk at the rear of the vehicle and a side camera is installed in the left side of the vehicle (for example, the left door mirror) and the rear of the vehicle 4 is imaged by the rear camera, for example, FIG. When an image as shown in FIG. 12 is obtained when, for example, an image as shown in FIG. 12 is obtained when the left side of the vehicle 4 is picked up by the side camera, the lines connecting the respective cameras and the obstacle 5 are perpendicular to each other. Since the distance in each direction can be displayed correctly, the distance (the number of dots on the screen) indicated by arrows a and b in FIGS. 11 and 12 is counted, and three squares are calculated based on this count value. By calculating using the theorem, the distance from the vehicle 4 to the obstacle 5 can be calculated.

  The correspondence between the configuration requirements of the embodiment and the configuration requirements in the claims is as follows. The imaging means is the camera 1, the overhead image creation means is the overhead conversion processing unit 21, and the image synthesis means is the image synthesis processing unit 23. The display means corresponds to the display 3 respectively.

It is a figure which shows the structure of the video display apparatus for vehicles which concerns on the Example of this invention. It is a figure which shows the positional relationship of the vehicle and an obstruction in the imaging time t. It is a figure which shows the positional relationship of the vehicle and an obstruction in imaging time (t-Δt). It is a flowchart which shows the procedure of the operation | movement which concerns on the Example of this invention. It is a figure which shows an example of the picked-up image in the state shown in FIG. It is a figure which shows the bird's-eye view image in the picked-up image of FIG. It is a figure which shows an example of the picked-up image in the state shown in FIG. It is a figure which shows the bird's-eye view image in the picked-up image of FIG. It is a figure which shows the bird's-eye view image which shifted the bird's-eye view image of FIG. 5 in consideration of the movement of a vehicle. 10 is a differential overhead image showing a difference between the overhead image shown in FIG. 6 and the overhead image shown in FIG. 9. It is an example of the bird's-eye view image corresponding to the picked-up image imaged with the rear camera. It is an example of the bird's-eye view image corresponding to the picked-up image imaged with the side camera.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Camera 2 ... Processing part 3 ... Display 4 ... Vehicle 5 ... Obstacle 21 ... Overhead view conversion process part 22 ... Image temporary storage part 23 ... Image composition process part

Claims (5)

Imaging means for imaging the periphery of the vehicle;
An overhead image creation means for creating an overhead image by converting the viewpoint of the image captured by the imaging means;
The position of the vehicle at the second time based on the amount of movement of the vehicle, the first overhead image created by converting the image captured by the imaging unit at the first time with the overhead image creation unit. And a second overhead image created by converting the image taken by the imaging means at the second time by the overhead image creating means, and An image synthesizing unit that extracts the difference between the images and creates a difference overhead image and superimposes the created difference overhead image on the second overhead image;
A vehicle video display device comprising: display means for displaying an overhead image synthesized by the image synthesis means. The vehicle image display device according to claim 1, wherein the image synthesizing unit detects an obstacle from the overhead image of the difference and performs an enhancement process on the obstacle. The vehicular video display apparatus according to claim 2, wherein the enhancement process surrounds an outline of the detected obstacle. The vehicular image display device according to claim 2 or 3, wherein the enhancement processing is performed so that the detected obstacle is displayed in a color different from that of other portions. A first step of imaging the periphery of the vehicle at a first time;
A second step of creating a first overhead image by converting the viewpoint of the image captured in the first step;
A third step of imaging the periphery of the vehicle at a second time;
A fourth step of creating a second overhead image by converting the viewpoint of the image captured in the third step;
A fifth step of moving the first bird's-eye view image created in the second step according to the position of the vehicle at a second time based on the amount of movement of the vehicle;
A sixth step of extracting a difference between the first overhead image moved in the fifth step and the second overhead image created in the fourth step to create a difference overhead image;
A seventh step of superimposing and synthesizing the difference overhead image created in the sixth step on the second overhead image;
And an eighth step of displaying the bird's-eye view image synthesized in the seventh step.

Images