JP2009044597A - Wide-angle image acquisition device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a wide-angle image acquisition device for obtaining a wide-angle image and distance information with simple constitution. <P>SOLUTION: Images picked up by TV cameras 11a and 11b are stored in image memories 12a and 12b respectively, projection conversion is applied to the images to confirm identical points of the images of the TV cameras 11a and 11b, and the images are synthesized to generate a wide-angle image. Then a display viewpoint of the wide-angle image is converted to form a wide-angle image which is viewed from a predetermined viewpoint. When the wide-angle image is generated, distances from the TV cameras 11a and 11b to an object recognized in the wide-angle image are calculated based upon information on overlap portions of the images, the size of the object is specified based upon the sizes of the images, the distances from the TV cameras 11a and 11b to the object recognized from the wide-angle image, and magnifications of the TV cameras 11a and 11b, and the object is displayed on a monitor 15 together with the wide-angle image. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a wide-angle image acquisition device.

2. Description of the Related Art Conventionally, a wide-angle imaging device that generates a wide-angle image using a plurality of images captured by a plurality of cameras and displays the image on a monitor is known (see the following patent publication).
JP 2006-25340 A

  However, in the invention disclosed in the above patent document, for example, the size of the target in the captured image cannot be known. In order to know the size of the target, it is necessary to measure the distance to the target. Therefore, in the invention disclosed in the above patent document, it is necessary to provide some distance sensor separately from the wide-angle imaging device. That is, when it is going to obtain | require distance information with the invention disclosed by the said patent document, it is necessary to provide a distance sensor with a wide angle imaging device, and a structure becomes complicated.

  The present invention has been made in view of such circumstances, and an object thereof is to obtain a wide-angle image acquisition apparatus that can obtain a wide-angle image and distance information with a simple configuration.

  In order to solve the above-mentioned problem, the invention according to claim 1 is obtained by at least one set of imaging devices including a plurality of cameras arranged close to each other so that respective angles of view partially overlap, and the plurality of cameras. And a distance from the camera to the object recognized by the wide-angle image based on information of overlapping portions of the images acquired by the plurality of cameras. And a distance measuring means for obtaining the characteristic.

  According to a second aspect of the present invention, in the wide-angle image acquisition device according to the first aspect of the present invention, the apparatus includes a size calculation unit that determines the size of the object based on information of overlapping portions of images acquired by the plurality of cameras. It is characterized by.

  According to a third aspect of the present invention, in the wide-angle image acquisition device according to the first aspect, the distance obtained by the distance measuring unit is displayed on the wide-angle image display unit together with the wide-angle image.

  According to a fourth aspect of the present invention, in the wide-angle image acquisition device according to the first aspect, an image showing a positional relationship between the object recognized from the wide-angle image and a vehicle on which the plurality of cameras are arranged is created. The image processing apparatus includes: a positional relationship image creating unit; and a display control unit that displays on the wide angle image display unit an image indicating a positional relationship between the object recognized from the wide angle image and the vehicle.

  According to a fifth aspect of the present invention, in the wide-angle image acquisition device according to the first to fourth aspects, the plurality of cameras are arranged at predetermined intervals in the vertical direction.

  According to the present invention, it is possible to obtain a wide-angle image acquisition apparatus that can obtain a wide-angle image and distance information with a simple configuration.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram illustrating a configuration of a wide-angle image acquisition apparatus according to the first embodiment of the present invention, and FIG. 2 is a diagram illustrating an example of a processing flow by a CPU. In FIG. 2, S1 to S8 indicate the steps of the process.

  This wide-angle image acquisition device includes TV cameras 11a and 11b, image memories 12a and 12b, an image processing unit 13, a distance calculation unit (ranging unit) 14, a monitor (wide-angle image display unit) 15, and a size calculation unit (size calculation). Means) 16. The image processing unit 13, the distance calculation unit 14, and the size calculation unit 16 are configured by a CPU, for example.

  The TV cameras 11a and 11b constitute an imaging device. The TV cameras 11a and 11b are arranged close to each other with a predetermined interval. The angle of view of each of the TV cameras 11a and 11b is about 135 °. When three TV cameras are used, the angle of view of each TV camera may be about 90 °.

  The image memory 12a stores image data captured by the TV camera 11a and A / D converted, and the image memory 12b stores image data captured by the TV camera 11b and A / D converted. The image memories 12a and 12b are semiconductor memories such as flash memories, for example.

  The image processor 13 combines the two images captured by the TV cameras 11a and 11b based on the image data stored in the image memory 12a and the image memory 12b to generate a wide-angle image.

  The distance calculation unit (ranging means) 14 calculates the distance from the TV cameras 11a and 11b to the object (not shown) based on the image data of the overlapping part of the two images captured by the TV cameras 11a and 11b. .

  The monitor 15 is an LCD (Liquid Crystal Display), for example, and displays a wide-angle image generated by the image processing unit 13, or displays the distance from the TV cameras 11a and 11b calculated by the distance calculation unit 14 to the object. To do.

  The size calculation unit 16 obtains the size of the object based on information to be described later in the overlapping portion of the images acquired by the TV cameras 11a and 11b.

  Next, the processing flow of this wide-angle image acquisition device will be described.

  First, images captured by the TV cameras 11a and 11b and A / D converted are stored in the image memories 12a and 12b, respectively (S1).

  Next, projective transformation of the image is performed to eliminate the distortion of the image (S2).

  Thereafter, the same point in the images of the TV cameras 11a and 11b is confirmed, and the image is as seen from a single viewpoint (S3).

  Next, a wide-angle image is created by synthesizing the images after the projective transformation (S4).

  Thereafter, the display viewpoint of the wide-angle image is converted to obtain a wide-angle image viewed from another viewpoint (S5).

  When the wide-angle image is synthesized, the distance from the TV cameras 11a and 11b to the object is calculated based on the information of the overlapping portions of the images (S6).

  Next, the size of the object is obtained based on the size of the image of the object, the distance to the object, and the magnification of the TV cameras 11a and 11b (S7).

  Thereafter, the distance and size to the object are displayed on the monitor 15 together with the wide-angle image (S8).

  According to this embodiment, the distance from the TV cameras 11a and 11b to the object is obtained based on the information of the overlapping portions of the images and displayed on the monitor 15 together with the wide-angle image. Distance information can be obtained. Moreover, it is difficult to design and manufacture, and an expensive wide-angle lens is not used, and an inexpensive lens with a narrow angle of view is used, so that the manufacturing cost can be reduced. Furthermore, if three TV cameras are used, an inexpensive TV camera with a narrow angle of view can be used, so that the manufacturing cost can be further reduced.

  FIG. 3 is a block diagram illustrating the configuration of a wide-angle image acquisition device according to the second embodiment of the present invention, FIG. 4 is a diagram schematically illustrating an arrangement example of an in-vehicle camera, and FIG. FIGS. 6A and 6B are diagrams showing an example of the TV camera arrangement. The same reference numerals are given to the portions common to the first embodiment, and the description thereof is omitted.

  This wide-angle image acquisition device includes vehicle-mounted cameras (imaging devices) 21A to 21D, an A / D converter 22, an image memory 23, an image processing unit 24, a distance calculation unit (distance calculation unit) 25, a positional relationship image creation unit 26, and a control unit. 27 and a monitor (wide-angle image display means) 28.

  The four in-vehicle cameras 21A to 21D are composed of TV cameras 11a and 11b (see FIG. 6), respectively. 4 and 6A, the TV cameras 11a and 11b are arranged side by side in the left-right direction, but the TV cameras 11a and 11b may be arranged side by side as shown in FIG. 6B. In FIG. 6A, the optical axes of the left and right lenses are in the same plane. In FIG. 6 (b), there are optical axes in parallel planes that are spaced apart by a predetermined interval, and each optical axis has a predetermined angle. In particular, if the exits of the lenses that are separated from each other in the vertical direction are matched, the parallax at the left and right positions is eliminated, and adjustment of the parallax at the left and right becomes unnecessary.

  The in-vehicle cameras 21A to 21D acquire images of the front, rear, left, and right sides of the vehicle Vh, respectively, and are disposed on the front, rear, left, and right sides of the vehicle Vh. The in-vehicle camera 21A is disposed at the front of the vehicle Vh, the in-vehicle camera 21B is disposed at the rear of the vehicle Vh, and the in-vehicle cameras 21C and 21D are disposed at the left side and the right side of the vehicle Vh, respectively. As shown in FIG. 6, the in-vehicle cameras 21 </ b> A to 21 </ b> D each have an angle of view of approximately 180 ° (the TV cameras 11 a and 11 b each have an angle of view of approximately 130 °), and an image around the vehicle Vh can be acquired. . In FIG. 4, hatched portions R1 to R8 are portions where the angles of view of the TV cameras 11a and 11b overlap. For example, the portion R1 comes to the portion S where the hatching in FIG.

  The image memory 23 is a semiconductor memory such as a flash memory. The image memory 23 stores image data around the vehicle Vh obtained by the in-vehicle cameras 21 </ b> A to 21 </ b> D based on an instruction from the control unit 27.

  Based on the image data stored in the image memory 23, the image processing unit 24 synthesizes the front-rear and left-right images of the vehicle Vh captured by the in-vehicle cameras 21A to 21D to generate a wide-angle image.

  The distance calculation unit 25 calculates the distance from the TV cameras 11a and 11b to the object (not shown) based on the image data of the overlapping portions of the images captured by the in-vehicle cameras 21A to 21D.

  Based on an instruction from the control unit 27, the positional relationship image creating unit 26 creates an image indicating the positional relationship between an object around the vehicle Vh and the vehicle Vh on which the in-vehicle cameras 21A to 21D are arranged. Note that the image data of the vehicle Vh may be stored in advance in the vehicle image memory 29, for example.

  The monitor 28 is, for example, an LCD. For example, the monitor 28 is arranged almost at the center of the center console so that the screen of the monitor 28 can be seen from the driver's seat. On the screen of the monitor 28, an image showing the positional relationship between the object around the vehicle Vh and the vehicle Vh is displayed.

  The control unit 27 is constituted by a microcomputer, for example.

  FIG. 7 is a diagram showing an example of an image displayed on the monitor screen.

  An image around the vehicle Vh acquired by each of the in-vehicle cameras 21A to 21D based on a command from the control unit 27 is A / D converted, and then sequentially stored in the image memory 23 at a predetermined timing.

  Based on the instruction from the control unit 27, the image processing unit 24 synthesizes the front-rear and left-right images of the vehicle Vh captured by the in-vehicle cameras 21A to 21D to generate a wide-angle image (360 °). In addition, the distance calculation unit 25 calculates the distance to the utility pole T1, the person T2, the bicycle T3, and the other vehicle T4 on the front, rear, left and right of the vehicle Vh.

  Based on an instruction from the control unit 27, the positional relationship image creation unit 26 synthesizes the wide-angle image generated by the image processing unit 24 and the image stored in the vehicle image memory 29 to position the vehicle Vh and the utility pole T1 and the like. An image showing the relationship is generated, and the image is displayed on the monitor 28 (see FIG. 7).

  In FIG. 7, the vehicle Vh, the utility pole T1, the human T2, the bicycle T3, and the other vehicle T4 are surrounded by a frame F. By viewing this image, the driver can easily recognize obstacles and people around the vehicle Vh.

  Although not shown, for example, image data of the vehicle Vh is stored in advance in the vehicle image memory 29, and this image and the bird's-eye view around the vehicle after the viewpoint conversion are combined and displayed on the monitor 28 as an overhead view. You can also. The display can be changed, for example, by selecting a predetermined item displayed on the monitor 28 with a finger or the like.

  According to this embodiment, the same effects as in the first embodiment and the following effects are obtained.

  Since a 360-degree wide-angle image around the vehicle Vh can be displayed, the positional relationship between the vehicle Vh and the utility pole T1 around the vehicle Vh can be grasped in detail, and safety when driving the vehicle Vh Can be improved.

  In this embodiment, a wide-angle image of 360 ° around the vehicle Vh is displayed on the monitor 28. However, for example, only the front or the rear of the vehicle Vh may be enlarged and displayed.

FIG. 1 is a block diagram illustrating the configuration of a wide-angle image acquisition apparatus according to the first embodiment of the present invention. FIG. 2 is a diagram illustrating an example of a processing flow by the CPU. FIG. 3 is a block diagram illustrating the configuration of a wide-angle image acquisition apparatus according to the second embodiment of the present invention. FIG. 4 is a diagram schematically illustrating an arrangement example of the in-vehicle camera. FIG. 5 is a diagram for explaining overlapping of the angle of view of the camera. 6A and 6B are diagrams showing examples of TV camera arrangement. FIG. 7 is a diagram showing an example of an image displayed on the monitor screen.

Explanation of symbols

  11a, 11b: TV camera, 13, 24: image processing unit, 14, 25: distance calculation unit (ranging unit), 15, 28: monitor (wide-angle image display unit), 16: size calculation unit (size calculation) Means), 21A to 21D: vehicle-mounted camera (imaging device), 26: positional relationship image creation unit.

Claims (5)

At least one set of imaging devices including a plurality of cameras arranged close to each other so that the respective angles of view partially overlap;
Wide-angle image display means for combining images acquired by the plurality of cameras and displaying as one wide-angle image;
A wide-angle image acquisition apparatus comprising: distance measuring means for obtaining a distance from the camera to an object recognized by the wide-angle image based on information of overlapping portions of images acquired by the plurality of cameras. .   The wide-angle image acquisition apparatus according to claim 1, further comprising: a size calculation unit that calculates a size of the object based on information of overlapping portions of images acquired by the plurality of cameras.   The wide-angle image acquisition apparatus according to claim 1, wherein the wide-angle image display unit displays the distance obtained by the ranging unit together with the wide-angle image.   Position relation image creating means for creating an image showing a positional relation between the object recognized from the wide-angle image and the vehicle on which the plurality of cameras are arranged, and the object recognized from the wide-angle image and the vehicle The wide-angle image acquisition apparatus according to claim 1, further comprising: a display control unit that causes the wide-angle image display unit to display an image that indicates a positional relationship with the wide-angle image display unit.   The wide-angle image acquisition apparatus according to claim 1, wherein the plurality of cameras are arranged at predetermined intervals in the vertical direction.

Images