JP2013141120A - Image display device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image display device capable of reliably displaying an object positioned in the posterior part of a mobile body when cameras are mounted in the rear part and rear side parts of the mobile body.SOLUTION: An image display device includes: a rear camera for imaging the posterior part of a vehicle; a rear left side camera and a rear right side camera for imaging the posterolateral parts of the vehicle; an image processing part for performing image processing to synthesize an image captured by the rear camera with images captured by the rear left side camera and the rear right side camera and also performing the synthesis of the images thereby to allow an area behind the position of an indication line set on the posterior side of the vehicle to be the captured image of the rear camera; and a display for displaying the image synthesized by the image processing part. Then, an object positioned in the posterior part of the vehicle is displayed in image processing to be performed with the use of the images obtained by imaging the posterior part of the vehicle and the obliquely posterior parts of the vehicle.

Description

  The present invention relates to an image display device.

  2. Description of the Related Art Conventionally, an image display apparatus that displays an object existing around a vehicle by combining images from a plurality of cameras mounted on the vehicle is known. Patent Document 1 (Japanese Patent Laid-Open No. 2001-055100) discloses a technology that includes a plurality of cameras and a monitor that displays camera images, and that specifies camera images to be subjected to image processing in accordance with the vehicle state. Yes. Patent Document 2 (Japanese Patent No. 3298851) discloses a technique for displaying camera images on a left monitor, a right monitor, and a central monitor at an appropriate timing.

JP 2001-055100 A Japanese Patent No. 3298851

  By the way, in the image display apparatus as described above, there are apparatuses that mount cameras having different mounting positions and orientations behind and behind a moving body such as a vehicle, and panorama synthesize the camera images captured by these cameras. . In this type of apparatus, when an image is generated by combining the camera images, a combination boundary for combining the images is set. Here, in the conventional image display apparatus, the composite boundary is set according to the distant background where the degree of coincidence of each image is high. Therefore, since the rear side image is displayed larger than the rear side of the moving body, there is a problem that an object located behind the moving body is not sufficiently displayed.

  The objective of this invention is providing the image display apparatus which can display reliably the object located behind a mobile body, when a camera is mounted in the back and back side of a mobile body.

  That is, the image display device according to the present invention includes a rear imaging unit that images the rear of the moving body, a rear side imaging unit that images the rear side of the moving body, and an image captured by the rear imaging unit, And an image process in which an image captured by the rear side imaging unit is combined, and an area rearward from a reference position set on the rear side of the movable body is captured by the rear imaging unit. Image processing means for combining the image processing means and display means for displaying the image combined by the image processing means.

  According to this aspect of the invention, the image processing unit synthesizes the images so that the rear side from the reference position is an image captured by the rear imaging unit. For this reason, in the image processing performed by combining the image behind the moving body and the image behind the oblique body, it is possible to reliably display the object located behind the moving body in the combined image.

  The image display apparatus according to the present invention further includes a viewpoint conversion unit that converts the viewpoint of the image captured by the rear imaging unit and the image captured by the rear side imaging unit, and the image processing unit is located behind the reference position. It is preferable that the synthesis is performed so that the region is an image whose viewpoint is converted to an image obtained when the rear side is imaged from the reference position by the viewpoint conversion unit. According to this invention, since the rear region from the reference position is converted into the rear image of the rear imaging means, it is possible to make the composite image a smooth image and to reliably display the object located behind. .

  Further, in the image display device according to the present invention, the imaging range of the rear imaging means and the imaging range of the rear side imaging means are obliquely behind the moving body and separated from the moving body by one lane or more laterally. It is preferable that the image processing means performs composition such that an image behind the intersecting position becomes an image captured by the rear imaging means. According to the present invention, since the imaging ranges intersect at a position separated by one or more lanes to the side, up to both adjacent lanes that are subject to lane change is the imaging range of the rear imaging means. Therefore, since the imaging ranges do not overlap in the rear area to the adjacent lanes, it is possible to reliably avoid the problem that the objects in this area are displayed twice. Therefore, an object located behind the moving body and in the vicinity of both adjacent lanes can also be reliably displayed.

  In the image display device according to the present invention, it is preferable that the image processing unit performs the synthesis so that the image captured by the rear imaging unit spreads sideways as it goes upward from the reference position. According to the present invention, since the composite image is generated so that the composite boundary is inclined, it is possible to reliably display an object that extends vertically near the boundary between the rear imaging region and the rear side imaging region. It becomes.

  In the image display device according to the present invention, it is preferable that the moving body is a vehicle, and the reference position is a position where the moving body can change lanes without being influenced by the rear vehicle. According to the present invention, the reference position is a position that is not affected by the lane change. Therefore, the driver of the vehicle can surely recognize the vehicle behind and can determine whether or not the lane change is possible.

  ADVANTAGE OF THE INVENTION According to this invention, when a camera is mounted in the back and back side of a moving body, it becomes possible to display reliably the object located behind the moving body.

It is a schematic block diagram of the image display apparatus which concerns on this embodiment. It is a figure for demonstrating the example of the image process in the image display apparatus of FIG. 3 is a flowchart illustrating image processing of the image display apparatus in FIG. 1. It is a figure for demonstrating the image processing which concerns on 1st Embodiment of this invention. It is a figure for demonstrating the image processing which concerns on 2nd Embodiment of this invention. It is a figure for demonstrating the image processing which concerns on 3rd Embodiment of this invention. It is a figure for demonstrating the image processing of the conventional image display apparatus.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In addition, the same code | symbol is used for the same element or the same equivalent element, and the overlapping description is abbreviate | omitted. As shown in FIG. 1, an image display device 1 according to the present embodiment is a device that is mounted on a moving body such as a vehicle 3 and captures the surroundings and performs image processing. Thus, the electronic mirror system performs image processing. This embodiment consists of the 1st-3rd embodiment from which a processing mode mutually differs. First, the image display device 1 of the first embodiment will be described.

  The image display device 1 includes a rear side left camera 11, a rear camera 12, a rear side right camera 13, and an image processing unit 2. The rear side left camera 11 and the rear side right camera 13 function as rear side imaging means for imaging the rear side of the vehicle 3. The photographing center of the rear side left camera 11 is located on the left side from the rear of the vehicle 3, and the photographing center of the rear side right camera 13 is located on the right side from the rear of the vehicle 3. The rear camera 12 functions as a rear imaging unit that images the rear of the vehicle 3. The imaging center of the rear camera 12 is located directly behind the vehicle 3. The rear side left camera 11, the rear camera 12, and the rear side right camera 13 transmit the captured camera images to the image processing unit 2.

  Further, the image display device 1 includes a display 14 and a mounting information holding unit 15. The display 14 has a function as display means for displaying the composite image generated by the image processing unit 2, and is an interface capable of inputting and outputting information between the image display device 1 and a passenger of the vehicle 3. The mounting information holding unit 15 records information regarding the rear side left camera 11, the rear camera 12, and the rear side right camera 13 (hereinafter, these cameras may be collectively referred to as “each camera”). Specifically, the mounting information holding unit 15 measures the position and inclination of each camera mounted on the vehicle 3 and records the measured values as camera mounting position information.

  The image processing unit 2 performs image processing by combining the rear image captured by the rear camera 12 and the rear side image captured by the rear side left camera 11 and the rear side right camera 13, and performs a predetermined reference. It functions as an image processing means for combining so that an image behind the position becomes an image captured by the rear camera 12. The image processing unit 2 includes a viewpoint conversion unit 21, an enlargement / reduction unit 22, and a clipping composition unit 23.

  The viewpoint conversion unit 21 functions as viewpoint conversion means for converting the viewpoint of the image captured by the rear camera 12 and the images captured by the rear side left camera 11 and the rear side right camera 13. The viewpoint conversion unit 21 performs viewpoint conversion of the camera image captured by each camera based on the camera mounting position information of the mounting information holding unit 15. The viewpoint conversion unit 21 performs conversion into a viewpoint image when each camera image is viewed from a specific viewpoint, and generates a virtual viewpoint image when a predetermined direction is viewed from a predetermined position. The enlargement / reduction unit 22 has a function of enlarging and reducing each camera image. For example, the enlargement / reduction unit 22 performs a process of enlarging or reducing the camera image subjected to the viewpoint conversion by the viewpoint conversion unit 21. The cutout synthesis unit 23 has a function of cutting out and synthesizing each camera image. For example, the combination boundary of each camera image is set, and the image is cut out and synthesized at the set synthesis boundary.

  Hereinafter, an example of image processing of the image processing unit 2 will be described with reference to FIG. First, as shown in part (a) of FIG. 2, the rear side left camera 11 and the rear side right camera 13 are provided in front of the rear camera 12. A case where the line a, the line b, and the object Y that is a cone are imaged will be described.

  First, the captured image of the rear side left camera 11 or the rear side right camera 13 is displayed on the display 14 as shown in FIG. 2B, for example, and the captured image of the rear camera 12 is, for example, shown in FIG. It is displayed on the display 14 as in part c). As shown in these drawings, in the image of the rear camera 12, the line a, the line b, and the object Y are displayed larger than the image of the rear side left camera 11 or the rear side right camera 13. At the same time, the line a and the line b are displayed below.

  Thus, since the positions and sizes of the line a and the line b are different from each other because the front and rear positions of the cameras are different, there is a problem that the composite image is not displayed smoothly. Therefore, the image processing unit 2 uses the camera mounting position information of the mounting information holding unit 15 to enlarge the portion below the road surface R, for example, to be a captured image of the rear camera 12 (portion (c) in FIG. 2). By using the original image for the portion above the road surface R, for example, a smooth composite image as shown in FIG. 2D is generated.

  Next, a conventional image display device will be described as a comparative example of the image display device 1 with reference to FIG. A conventional image display device is mounted on a vehicle 103 and includes a rear side left camera 111, a rear camera 112, and a rear side right camera 113. The rear side left camera 111, the rear camera 112, and the rear side right camera 113 are cameras corresponding to the rear side left camera 11, the rear camera 12, and the rear side right camera 13, respectively.

  In the conventional image display device, panoramic synthesis is performed on the camera images of the rear side left camera 111, the rear camera 112, and the rear side right camera 113 to generate an image around the vehicle 103. At the time of panorama composition, a composition boundary of the imaging range of each camera is set. Here, in the conventional image display apparatus, the composite boundary is set according to the distant background where the degree of coincidence of each imaging range is high. That is, for the background at infinity, the imaging range A and the imaging range C of the rear side left camera 111 and the rear side right camera 113 are smoothly joined, for example, as shown in part (b) of FIG. On the other hand, since the imaging range B of the rear camera 112 is only a road surface image, there is a problem that an object behind the vehicle 103 is not displayed.

  Specifically, in the conventional image display device, for example, as shown in the plan view of part (a) of FIG. 7, the imaging range A extends diagonally to the left and the imaging range C extends diagonally rearward to the right and the imaging range B is rearward. The composite boundary is set so as to extend to. However, for example, as shown in part (b) of FIG. 7, the image displayed on the display 14 is only the image of the imaging range A and the imaging range C above the road surface, and the image of the imaging range B is not included. For this reason, the object located behind the vehicle 103 is not displayed, for example, the problem that the driver of the vehicle 103 cannot recognize the rear vehicle occurs.

  Therefore, in the image display device 1 according to the first embodiment, the above-described problem is solved by performing the synthesis so that the rear of the predetermined camera position is the image of the rear camera 12 by the image processing unit 2. Hereinafter, an example of the operation of the image display apparatus 1 that performs such a composition process will be described with reference to FIG. FIG. 3 is a flowchart of image processing in which the image processing unit 2 generates a combined image by combining the camera images of the cameras. The process shown in FIG. 3 may be performed every predetermined time or may be performed when the driver performs an image display operation.

  First, in step S10 (hereinafter referred to as “S10”, the same applies to other steps), the image processing unit 2 executes a process of reading a camera image captured by each camera. At this time, for example, each camera image is stored in a storage device in the image processing unit 2. Then, the process proceeds to S12, and the viewpoint conversion process is executed by the viewpoint conversion unit 21. Specifically, viewpoint conversion of each image is performed based on the camera mounting position information of the mounting information holding unit 15. At this time, for example, as shown in part (b) of FIG. 4, a reference line L is set, and a viewpoint conversion image is generated by converting the viewpoint of the rear camera 12 to a viewpoint when viewing the back from the reference line L.

  Here, the reference line L functions as a reference position set on the rear side of the moving body. For example, when the vehicle 3 changes the lane, the lane can be changed without being influenced by the rear vehicle. Can be provided. That is, for example, the safety distance is determined based on the speed of the vehicle 3, the speed of the rear vehicle, or the size of the vehicle body of the vehicle 3 and the rear vehicle, and the reference line L is set at a position separated backward by the safety distance. The reference line L is a line used as a guideline when cutting out an image, and may not be displayed on the display 14.

  After S12, the process proceeds to S14, and the image synthesizing unit 23 executes the image clipping process. At this time, for example, an image above the reference line L set in S12 is cut out. Thereafter, the process proceeds to S16, and the image composition processing is executed by the cutout composition unit 23. Specifically, the viewpoint conversion image generated in S12 is combined instead of the image cut out in S14 to generate a combined image as shown in, for example, part (b) of FIG. The plan view of the imaging range of each camera in this composite image is, for example, as shown in FIG. 4A, and the difference from the FIG. 7A portion is that the imaging range B of the rear camera 12 is behind the reference line L. Is the point that spreads in the vehicle width direction toward the rear.

  After S16, the process proceeds to S18, and the composite image display process is executed by the image processing unit 2. At this time, a synthesized image as shown in FIG. 4B synthesized in S16 is displayed on the display 14, and the series of processing ends.

  As described above, according to the image display device 1 of the first embodiment, the rear camera 12 that captures the rear of the vehicle 3, the rear left camera 11 and the rear right camera that capture the rear side of the vehicle 3. 13 and the image captured by the rear camera 12 and the images captured by the rear side left camera 11 and the rear side right camera 13 are combined to perform image processing. And an image processing unit 2 that performs synthesis so as to be an image captured by the image processing unit 12.

  As described above, in the first embodiment, the image processing unit 2 combines the images so that the rear of the reference line L becomes the image of the rear camera 12, so that the images obtained by capturing the rear and oblique rear of the vehicle 3 are combined. In the image processing performed as described above, an object located behind the vehicle 3 can be reliably displayed, and the driver can reliably recognize an object located behind the vehicle 3.

  In the first embodiment, the image processing unit includes a viewpoint conversion unit 21 that performs viewpoint conversion of an image captured by the rear camera 12 and images captured by the rear side left camera 11 and the rear side right camera 13. 2 performs synthesis so that the area behind the reference line L becomes an image whose viewpoint is converted to an image when the viewpoint conversion unit 21 captures the rear of the reference line L. Therefore, since the rear region from the reference line L is converted into the rear image of the rear camera 12, the composite image can be made a smooth image and the object located behind can be displayed reliably.

  In the first embodiment, the reference line L is provided at a position where the vehicle 3 can change the lane without being influenced by the rear vehicle. As described above, the reference line L is located behind the vehicle 3 and is not affected by the rear vehicle, so that the driver of the vehicle 3 can surely recognize the rear vehicle and change the lane. A determination can be made as to whether it is possible.

  Next, an image display apparatus according to the second embodiment will be described with reference to FIG. The second embodiment has the same configuration as the image display device 1 of the first embodiment, but the image processing by the image processing unit 2 is different. Specifically, the method of viewpoint conversion and composite boundary setting in S12 in FIG. Is different.

  In the second embodiment, after the image reading process of S10 is performed as in the first embodiment, the reference line L is positioned closer to the vehicle 3 than the first embodiment in S12, specifically, the lane l ( The distance is set so as not to affect the change (see part (a) of FIG. 5) (for example, within 15 m from the rear end of the vehicle 3). Then, the image processing unit 2 generates a viewpoint-converted image in which the viewpoint of the rear camera 12 is converted to a viewpoint when the rear is viewed from the reference line L.

  At this time, the image processing unit 2 sets the imaging range A to the left rear of the vehicle 3, the imaging range C to the right rear of the vehicle 3, and the imaging range B to the vehicle 3 as shown in FIG. The trapezoidal shape extends from the rear, and the intersections of the imaging range A and the imaging range B and the intersections of the imaging range B and the imaging range C are both located away from the vehicle 3 by one lane or more laterally. Set the imaging range. That is, for example, a composite image as shown in part (b) of FIG. 5 is generated. Thereafter, the display process on the display 14 in S18 is executed through the processes in S14 and S16, and then the series of processes is completed.

  As described above, in the second embodiment, the imaging range B, the imaging range A, and the imaging range C intersect with each other at a position that is diagonally rearward of the vehicle 3 and separated from the vehicle 3 by one lane or more laterally. Then, the image processing unit 2 performs synthesis so that the rear side from the intersecting position is an image captured by the rear camera 12. Therefore, the area up to both adjacent lanes 1 in the rear of the vehicle 3 is the imaging range B of the rear camera 12, and the imaging ranges do not overlap in this area, so that the problem of the object being displayed twice is surely avoided. It becomes possible to do. Therefore, in the image display device of the second embodiment, the same effect as that of the first embodiment can be obtained, and the driver can reliably recognize an object located behind the vehicle 3 and in the adjacent lane l. be able to.

  Next, an image display apparatus according to a third embodiment will be described with reference to FIG. The third embodiment has the same configuration as the image display devices 1 of the first and second embodiments, but the image composition processing by the image processing unit 2 is different. Specifically, the processing of S12 in FIG. 3 is different.

  In the third embodiment, after performing the image reading process of S10 as in the first and second embodiments, in the process of S12, the image processing unit 2 performs, for example, a reference line as shown in part (b) of FIG. In the image behind L, the composite boundary is set so that the imaging range B expands laterally as it goes upward. At this time, for example, an imaging range as shown in the plan view of part (a) of FIG. 6 is set. After the image composition process is completed in this way, the process proceeds to S14, S16, and S18, and after the composite image display process is executed, the series of processes ends.

  As described above, in the third embodiment, the image processing unit 2 performs synthesis so that the image behind the reference line L becomes an image that spreads sideways as it goes upward. Therefore, in the third embodiment, the same effect as in the first embodiment is obtained, and the composite boundary is set to be inclined, so that the upper and lower sides of the boundary between the imaging range B and the imaging range A and the imaging range C are moved up and down. It is also possible to reliably display the object X (see FIG. 6) that extends and exists. Therefore, the driver can reliably recognize a pedestrian or the like located on the rear side of the vehicle 3.

  As described above, the present embodiment describes the embodiment of the image display device according to the present invention, and the image display device according to the present invention is not limited to the one described in the present embodiment. The image display device according to the present invention may be obtained by modifying the image display device according to the present embodiment or applying it to other devices without changing the gist described in each claim. The image display device according to the present invention includes an image display device having at least one of the functions of the first embodiment, the second embodiment, and the third embodiment, and an image display device having both of these functions. It is.

  For example, in the above embodiment, an example in which the rear side left camera 11 is provided on the left side of the vehicle 3, the rear camera 12 on the rear side, and one rear side right camera 13 on the right side is described. However, the number and position of each camera are not limited to the above and can be changed as appropriate.

  Moreover, in the said embodiment, although the vehicle 3 was mentioned as an example as a moving body, you may make it apply to a robot etc. instead of the vehicle 3, and in short, it is applicable if it is a moving body which moves a road surface. It is.

  DESCRIPTION OF SYMBOLS 1 ... Image display apparatus, 2 ... Image processing part, 3 ... Vehicle 3, 11 ... Rear side left camera, 12 ... Rear camera, 13 ... Rear side right camera, 14 ... Display, 15 ... Mount information holding part, 21 ... viewpoint conversion unit, 22 ... enlargement / reduction unit, 23 ... clipping composition unit, A, B, C ... imaging range, L ... guide line.

Claims (5)

A rear imaging means for imaging the rear of the moving body;
Rear side imaging means for imaging the rear side of the moving body;
An image processing unit that combines an image captured by the rear imaging unit and an image captured by the rear side imaging unit, and is a region behind a reference position set on the rear side of the moving body Image processing means for synthesizing the image so as to be an image captured by the rear imaging means
Display means for displaying an image synthesized by the image processing means;
Comprising
Image display device. A viewpoint conversion means for converting the viewpoint of the image captured by the rear imaging means and the image captured by the rear side imaging means;
The image processing means performs synthesis so that a region behind the reference position becomes an image whose viewpoint is converted to an image when the viewpoint conversion means images the rear from the reference position.
The image display device according to claim 1. The imaging range of the rear imaging means and the imaging range of the rear side imaging means intersect at a position that is diagonally behind the moving body and separated from the moving body by one lane or more laterally,
The image processing means performs synthesis so that an image behind the intersecting position becomes an image captured by the rear imaging means.
The image display device according to claim 1. The image processing means performs synthesis so that an image picked up by the rear image pickup means spreads sideward as it goes upward from the reference position.
The image display apparatus of any one of Claims 1-3. The moving body is a vehicle,
The reference position is a position where the moving body can change lanes without being affected by a rear vehicle.
The image display apparatus of any one of Claims 1-4.

Images