JP2007008401A - Device and method for detecting position of image pickup device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a "device and method for detecting a position of an image pickup device" for determining a mounting position of a camera even when a versatile camera whose mounting position is not decided is mounted to one's own vehicle. <P>SOLUTION: The way how each image is overlapped is determined by an image determination part 4 by analyzing feature points of images captured by a plurality of the cameras 2a-2d. The mounting position is specified by a position detection part 5 on the basis of the determination results. Thus, it is determined by which camera each of the images captured by a plurality of the cameras 2a-2d is captured. Consequently, it is possible to determine the mounting position even when using the versatile camera whose mounting position is not decided. It is also possible to facilitate the mounting work since it is not necessary to select a camera corresponding to the mounting position or to execute the setting work of positional information when mounting a plurality of the cameras 2a-2d. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a position detection apparatus for an image pickup apparatus and a position detection method for the image pickup apparatus that detect attachment positions of a plurality of image pickup apparatuses attached around the own vehicle in order to take an image around the own vehicle.

  Conventionally, a technique has been proposed in which a plurality of cameras are installed in the host vehicle, images around the host vehicle are captured by the plurality of cameras, and the captured images are displayed on a display device such as a display installed in the host vehicle. Yes. By checking the image displayed on the display device of the host vehicle, the driver can grasp the distance between the host vehicle and the obstacle, and perform a traffic accident such as a collision between the host vehicle and the obstacle or a rear-end collision with another vehicle. Drive to prevent.

  In addition, a plurality of cameras are installed in the host vehicle, and an image viewed from a virtual viewpoint above the host vehicle is generated based on images taken by the plurality of cameras, and the image is displayed on a display device such as a display. Thus, a technique for monitoring the periphery of the host vehicle has been proposed.

  By the way, when a plurality of cameras are installed in the host vehicle, a top view image cannot be generated unless the position where each camera is attached is specified. Therefore, conventionally, as shown in FIG. 5A, when the plurality of cameras 20a to 20d are attached, the plurality of cameras 20a to 20d and a plurality of image input units 21a to 21a for acquiring images captured by these cameras 20a to 20d. 21d is provided in a one-to-one correspondence, and the plurality of cameras 20a to 20d and the image input units 21a to 21d are connected by a video cable dedicated to each mounting position.

  However, such a method has a problem that the image input units 21a to 21d corresponding to the cameras 20a to 20d must be prepared. Further, the image input units 21a to 21d are configured by an ECU (Electric Control Unit), but the cameras 20a to 20d and the image input units 21a to 21d are connected by a dedicated video cable. There was a problem of increasing.

  In order to solve this problem, for example, as shown in FIG. 5B, when attaching the plurality of cameras 20a to 20d, an image input unit 23 for acquiring images taken by the plurality of cameras 20a to 20d is provided. A technique for providing a bus connection between a plurality of cameras 20a to 20d and the image input unit 23 has been proposed.

  However, in such a method, in order to allow the ECU to identify where the plurality of cameras 20a to 20d are attached, it is necessary to set position information when the cameras are attached. In order to eliminate such work, dedicated ID (Identification) is set for each of the plurality of cameras 20a to 20d, and when the images captured by the plurality of cameras 20a to 20d are flowed on the bus, this ID is also set. A method of flowing together is conceivable. However, this method has a problem that since the cameras 20a to 20d are assigned with dedicated IDs, the cameras are dedicated to the respective positions, and the versatility of the cameras is lost.

By the way, when a plurality of cameras are installed in the host vehicle, images around the host vehicle are captured by the plurality of cameras, and the captured images are displayed on a display device such as a display installed in the host vehicle. There has been proposed a technique for displaying where a camera that captures the image displayed on the vehicle is installed (for example, Patent Document 1).
JP 2004-56683 A

  However, in this patent document 1, an image around the host vehicle is photographed using a dedicated camera whose attachment position is determined, and the camera at which position the image displayed on the display device is photographed. Is merely an indication. That is, the conventional technique described in Patent Document 1 is a technique based on the premise that it is known in advance where each camera is mounted, and the camera mounting position cannot be determined.

  The present invention has been made to solve such a problem, and even when a versatile camera whose mounting position is not determined is mounted on a vehicle, the user can perform position information setting work. It is an object to make it possible to determine the mounting position of a camera.

  In order to solve the above-described problems, the present invention inputs a plurality of images taken by a plurality of cameras, analyzes the feature points of each input image, determines how to overlap each image, and the determination result Based on the above, the mounting positions of a plurality of cameras are specified.

  According to the present invention configured as described above, it is determined by which camera the images are captured based on how the plurality of images captured by the plurality of cameras overlap with other images. Therefore, even when a versatile camera whose attachment position is not determined is attached to the vehicle, the attachment position of the camera can be determined. Therefore, when attaching a camera, it is not necessary to select a dedicated camera corresponding to the attachment position or to perform a setting operation of position information, and the camera attachment operation is facilitated.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram illustrating a configuration example of an image display device including a position detection device 1 according to the present embodiment. In FIG. 1, reference numeral 2 denotes an image pickup apparatus, which includes a first camera 2a, a second camera 2b, a third camera 2c, and a fourth camera 2d for photographing the periphery of the host vehicle. Here, no position ID or the like is set for the first camera 2a to the fourth camera 2d, and the first camera 2a to the fourth camera 2d can be attached to any position of the host vehicle. . Further, the first camera 2a to the fourth camera 2d respectively photograph the front, left side, right side, and rear when viewed from the host vehicle.

  As shown in FIG. 2, the front camera 12 a (any one of the first camera 2 a to the fourth camera 2 d) disposed in front of the host vehicle 13 captures a shooting range 14 a in front of the host vehicle 13. To do. Further, the left side camera 12b (any one of the remaining three cameras) disposed on the left side of the host vehicle 13 captures the shooting range 14b on the left side of the host vehicle 13. Also, the right side camera 12c (any one of the remaining two cameras) arranged on the right side of the host vehicle 13 captures the shooting range 14c on the right side of the host vehicle 13. Further, the rear camera 12d (remaining one camera) disposed behind the host vehicle 13 captures a shooting range 14d behind the host vehicle 13.

  Here, the left side camera 12b and the right side camera 12c are positioned higher than the front camera 12a and the rear camera 12d (for example, the left side camera 12b and the right side camera 12c are 1 meter high, 12a and rear camera 12d are mounted at a height of 0.6 meters), the shooting ranges 14b and 14c are wider than the shooting ranges 14a and 14d. Further, the left side camera 12 b and the right side camera 12 c are attached to the front side of the host vehicle 13. Therefore, there is an area 15a where the shooting range 14a of the front camera 12a and the shooting range 14b of the left side camera 12b overlap. Further, there is a region 15b where the shooting range 14a of the front camera 12a and the shooting range 14c of the right side camera 12c overlap. Further, there is no overlapping area between the shooting range 14b of the left side camera 12b, the shooting range 14c of the right side camera 12c, and the shooting range 14d of the rear camera 12d.

  Reference numeral 3 denotes an image input unit, to which a first camera 2a to a fourth camera 2d are connected via a bus, and images taken by the first camera 2a to the fourth camera 2d are input via the bus.

  Reference numeral 4 denotes an image determination unit which inputs images taken by the first camera 2 a to the fourth camera 2 d from the image input unit 3. Further, the image determination unit 4 analyzes feature points between images taken by the first camera 2a to the fourth camera 2d. Specifically, the image determination unit 4 analyzes feature points in a region within a predetermined range on both sides of the shooting range of each image, and whether there is a feature point having a similarity greater than or equal to a certain value between the images. If it exists, it is determined in which position of each image it exists. Here, the feature point is a portion where the brightness and color tone of the image is different from other portions, and the degree of similarity can be determined by comparing the shape and the like. Then, it is determined that feature points having a similarity equal to or higher than a certain value are “match”.

  And the image determination part 4 determines how to overlap each image based on the analysis result of the feature point in each image. That is, as shown in FIG. 2, the image of the shooting range 14a of the front camera 12a and the image of the shooting range 14b of the left side camera 12b overlap in the region 15a, and the image of the shooting range 14a of the front camera 12a and the image of the right side camera 12c. The image of the shooting range 14c overlaps in the region 15b, and there is an overlap between the image of the shooting range 14b of the left side camera 12b, the image of the shooting range 14c of the right side camera 12c, and the image of the shooting range 14d of the rear camera 12d. do not do. The image determination unit 4 can determine which image has which overlap or whether there is no overlap by seeing where the matching feature points are.

  Reference numeral 5 denotes a position detection unit, which identifies which of the front cameras 12a to 12d is taken by each image based on the determination result of the image determination unit 4. Specifically, the position detection unit 5 checks whether there is an image of a determination result that has a feature point that matches between other images in the left and right regions. When such an image exists, the position detection unit 5 determines that the image is an image taken by the front camera 12a. In addition, the position detection unit 5 determines that the image of the determination result that only the right region has a matching feature point between the other images is an image taken by the left side camera 12b. In addition, the position detection unit 5 determines that the image of the determination result that the feature point matches between the other images only in the left region is an image captured by the right side camera 12c. Further, the position detection unit 5 determines that the image of the determination result that there is no matching feature point between the other images is an image captured by the rear camera 12d.

  For example, as illustrated in FIG. 3, the image determination unit 4 matches the feature point 16a in the left region of the first image A with the feature point 16c in the right region of the second image B. It is determined that the feature point 16b in the right region of A matches the feature point 16d in the left region of the third image C. Thereby, the image determination unit 4 determines that the first image A has an overlap with other images (the second image B and the third image C) in the left and right regions. The position detection unit 5 determines that the first image A is an image taken by the front camera 12a according to the determination result by the image determination unit 4.

  Further, the image determination unit 4 has no feature point that matches between the other images in the left region of the second image B, and the feature point 16c of the right region matches the feature point 16a of the first image A. Judge that. Thereby, the image determination unit 4 determines that the second image B overlaps with another image (first image A) only in the right region. The position detection unit 5 determines that the second image B is an image taken by the left side camera 12b according to the determination result by the image determination unit 4.

  Further, the image determination unit 4 does not have a matching feature point between the other images in the right region of the third image C, and the feature point 16d of the left region matches the feature point 16b of the first image A. Judge that. Thereby, the image determination unit 4 determines that the third image C overlaps with another image (first image A) only in the left region. The position detection unit 5 determines that the third image C is an image taken by the right side camera 12c according to the determination result by the image determination unit 4.

  Further, the image determination unit 4 determines that there is no matching feature point between the other images in the right region and the left region of the fourth image D. Thereby, the image determination unit 4 determines that the fourth image D does not overlap with the other images (the first image A to the third image C). The position detection unit 5 determines that the fourth image D is an image taken by the rear camera 12d according to the determination result by the image determination unit 4. In the present embodiment, the feature points 16a and 16c and the feature points 16b and 16d have different shapes, but are not limited thereto. For example, all feature points may have the same shape.

  As shown in FIG. 2, the orientation of each image is shifted by 90 degrees. Therefore, as shown in FIG. 3, the direction of the feature point 16a of the first image A and the feature point 16c of the second image B existing in the region 15a is shifted by 90 degrees. Similarly, the feature point 16b of the first image A and the feature point 16d of the third image C existing in the region 15b are shifted by 90 degrees. Therefore, when analyzing feature points for an image (for example, the first image A), pattern matching is performed with the other images (second image B to fourth image D) rotated 90 degrees to the left. By doing this, the similarity between the feature point 16a and the feature point 16c can be determined more accurately. In addition, pattern matching is performed with the other images (second image B to fourth image D) rotated 90 degrees to the right, thereby more accurately determining the similarity between the feature point 16b and the feature point 16d. can do.

  In addition, the position detection unit 5 displays position information indicating which of the front camera 12a to the rear camera 12d is used for the first image A to the fourth image D as the first camera 2a to the first camera 2a. The information is stored in the position storage unit 6 together with information for identifying the four cameras 2d. As described above, the position storage unit 6 stores position information indicating which of the front camera 12a to the rear camera 12d is the first camera 2a to the fourth camera 2d. Therefore, after the position detection of the imaging device 2 is performed, the image input to the image determination unit 4 is an image generation unit which will be described later in a state where the photographed camera is specified by the position information stored in the position storage unit 6. 7 is output.

  The image generation unit 7 includes an image processing unit 8, a mapping table 9, and an image storage unit 10. The image processing unit 8 generates a background image when the periphery of the host vehicle 13 is viewed from above according to the conversion information stored in the mapping table 9. In this state, an image when the host vehicle 13 is viewed from above is not formed.

  The mapping table 9 describes the correspondence between the pixel data of the image of each camera to which the position information is added by the image determination unit 4 and the pixel data of the background image when the periphery of the host vehicle 13 is viewed from the upper virtual viewpoint. It is a table and describes conversion information indicating which pixel of a background image corresponds to a certain pixel of a photographed image. The image storage unit 10 stores host vehicle image data indicating an image of the host vehicle 13 as viewed from above, and the host vehicle image data is read by the image processing unit 8 as necessary.

  The image processing unit 8 generates a background image according to the position information read from the position storage unit 6. For example, an image photographed by the front camera 12a is arranged in front of the host vehicle 13, an image photographed by the left side camera 12b is arranged on the left side of the host vehicle 13, and an image photographed by the right side camera 12c is arranged. A background image is generated such that it is arranged on the right side of the host vehicle 13 and an image photographed by the rear camera 12d is arranged behind the host vehicle 13. In addition, the image processing unit 8 reads out own vehicle image data indicating an image of the host vehicle 13 as viewed from above from the image storage unit 10, and a predetermined position of the background image (for example, a position where the host vehicle 13 exists in the background image). To generate an image. Reference numeral 11 denotes a display device, which displays an image generated by the image processing unit 8.

  Next, the operation of the position detection device 1 of the imaging apparatus and the position detection method of the imaging apparatus according to the present embodiment will be described. FIG. 4 is a flowchart illustrating the operation of the position detection device 1 of the imaging apparatus and the position detection method of the imaging apparatus according to the present embodiment. In FIG. 4, the first camera 2 a to the fourth camera 2 d attached to the host vehicle 13 capture the periphery of the host vehicle 13 and output the captured image to the image input unit 3.

  In the image determination unit 4, images captured by the first camera 2 a to the fourth camera 2 d are acquired from the image input unit 3. Then, the image determination unit 4 analyzes the feature points of each image acquired from the image input unit 3, and examines how the images overlap based on the analysis result of the feature points in each image (step S1). The image determination unit 4 extracts a certain image (for example, an image photographed by the first camera 2a), and determines whether or not the image overlaps with other images in the left and right regions (step S2). ). If the image determination unit 4 determines that there is an overlap with other images in the left and right regions (YES in step S2), the position detection unit 5 is an image taken by the front camera 12a. That is, it is determined that the first camera 2a is the front camera 12a (step S3). Then, the process jumps to step S9.

  On the other hand, when the image determination unit 4 determines that there is no overlap with other images in the left and right regions (NO in step S2), the image determination unit 4 determines that the image is only in the right region. It is checked whether or not there is an overlap with the image (step S4). If the image determination unit 4 determines that there is an overlap with another image only in the right region (YES in step S4), the position detection unit 5 captures the image by the left side camera 12b. It is determined that the image is an image, that is, the first camera 2a is the left side camera 12b (step S5). Then, the process jumps to step S9.

  On the other hand, when image determination unit 4 determines that there is no overlap with other images only in the right region (NO in step S4), image determination unit 4 determines that the image is only in the left region. It is checked whether or not there is an overlap with another image (step S6). When the image determination unit 4 determines that there is an overlap with other images only in the left region (YES in step S6), the position detection unit 5 captures the image with the right side camera 12c. It is determined that the image is an image, that is, the first camera 2a is the right side camera 12c (step S7). Then, the process jumps to step S9.

  On the other hand, when image determination unit 4 determines that there is no overlap with other images only in the left region (NO in step S6), image determination unit 4 captures the image with rear camera 12d. That is, it is determined that the first camera 2a is the rear camera 12d (step S8).

  In step S9, the position detection unit 5 displays position information indicating which of the front camera 12a to the rear camera 12d is the first camera 2a to the fourth camera 2d, the first camera 2a to the fourth camera. The information for identifying 2d is stored in the position storage unit 6 (step S9). Then, the position detection unit 5 checks whether or not the attachment positions have been determined for all the cameras (first camera 2a to fourth camera 2d) (step S10). If position detection unit 5 determines that positions have been determined for all cameras (YES in step S10), the process ends.

  On the other hand, when the position detection unit 5 determines that the positions of all the cameras have not been determined (NO in step S10), the next image (for example, an image photographed by the second camera 2b) is selected. (Step S11). Then, the process returns to step S2. In the present embodiment, the image taken by the front camera 12a is first identified, but the present invention is not limited to this. For example, an image captured by the left side camera 12b, the right side camera 12c, and the rear camera 12d may be specified first, and then another image may be specified.

  As described above in detail, in the present embodiment, images captured by the first camera 2a to the fourth camera 2d are acquired, and feature points of the images are analyzed by the image determination unit 4 to overlap each image. Is judging. In addition, the position detection unit 5 identifies which image was captured by which camera based on the determination result of the image determination unit 4 regarding how the images overlap. Thereby, since the image image | photographed with the 1st camera 2a has an overlap with another image in the area | region on either side, it can determine with the 1st camera 2a being the front camera 12a. In addition, since the image captured by the second camera 2b has an overlap with other images only in the right region, it can be determined that the second camera 2b is the left side camera 12b. In addition, since the image captured by the third camera 2c overlaps with other images only in the left region, it can be determined that the third camera 2c is the right side camera 12c. In addition, since there is no overlap between the image captured by the fourth camera 2d and another image, it can be determined that the fourth camera 2d is the rear camera 12d. As described above, even when the versatile cameras 2a to 2d whose attachment positions are not determined are attached to the host vehicle 13, the attachment positions can be automatically determined. Therefore, when attaching the cameras 2a to 2d, there is no need to select a camera corresponding to the attachment position or to perform position information setting work, and the camera 2a to 2d can be easily attached.

  In the present embodiment, there is an area 15a where the shooting range 14a of the front camera 12a and the shooting range 14b of the left side camera 12b overlap, and the shooting range 14a of the front camera 12a and the shooting range 14c of the right side camera 12c. Although the overlapping area 15b is provided, the present invention is not limited to this. For example, the left side camera 12b and the right side camera 12c are installed near the rear of the host vehicle 13, and the rear camera 12d has an area where the shooting range 14d of the rear camera 12d and the shooting range of the left side camera 12b overlap. The imaging range 14d and the imaging range of the right side camera 12c may have an overlapping area. In such a case, the position detection unit 5 determines that an image having a feature point matching the other image in the left and right regions is an image captured by the rear camera 12d. Further, the position detection unit 5 determines that an image having an overlap with another image in only the left region is an image taken by the left side camera 12b. Further, the position detection unit 5 determines that an image having an overlap with another image only in the right region is an image taken by the right side camera 12c. Further, the position detection unit 5 determines that an image that does not overlap with other images is an image taken by the front camera 12a.

  Moreover, in this embodiment, although the periphery of the own vehicle 13 is image | photographed with four cameras, it is not limited to this. For example, three cameras that photograph the front and side of the host vehicle 13 may be used. Also, when the shooting range of the rear camera 12d and the shooting range of the left side camera 12b have an overlapping area, and the shooting range of the rear camera 12d and the shooting range of the right side camera 12c have an overlapping area. May use three cameras for photographing the back and sides of the host vehicle 13.

  In the present embodiment, the first camera 2a to the fourth camera 2d that do not have position information are bus-connected to the image input unit 3, but the present invention is not limited to this. For example, the first camera 2a to the fourth camera 2d may be connected one to one to four image input units that do not have position information.

  In addition, each of the above-described embodiments is merely an example of actualization in carrying out the present invention, and the technical scope of the present invention should not be construed in a limited manner. In other words, the present invention can be implemented in various forms without departing from the spirit or main features thereof.

  INDUSTRIAL APPLICABILITY The present invention is useful for an image display device that displays an image of the periphery of the host vehicle captured by a plurality of imaging devices attached around the host vehicle.

It is a block diagram which shows the structural example of the image display apparatus containing the position detection apparatus of the imaging device by this embodiment. It is a figure which shows the example of arrangement | positioning of each camera by the position detection apparatus of this embodiment, and the example of the imaging range of each camera. It is a figure which shows the example of the image image | photographed with each camera by the position detection apparatus of this embodiment. It is a flowchart which shows the operation | movement of the position detection apparatus of the imaging device by this embodiment, and a position detection method. It is a block diagram which shows the example of a connection of the conventional camera.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Position detection apparatus 2 Imaging device 2a 1st camera 2b 2nd camera 2c 3rd camera 2d 4th camera 3 Image input part 4 Image determination part 5 Position detection part 6 Position memory | storage part 7 Image generation part 12a Front camera 12b Left side camera 12c Right side camera 12d Rear camera 13 Own vehicle

Claims (3)

A first imaging device that captures a first direction and captures a first image in order to capture an image of the surroundings of the host vehicle, a second direction on the left side of the first direction, and the first direction. A second imaging device that acquires a second image that partially overlaps one image and a third direction on the right side of the first direction, and a portion that overlaps the first image A position detection device for an image pickup device for detecting a mounting position of a third image pickup device for acquiring a third image,
The first to third images captured by the first to third imaging devices are input, and feature points of the input images are analyzed to determine how the images overlap. An image determination unit;
A position detection unit that specifies an attachment position of the first imaging device to the third imaging device based on how the images determined by the image determination unit overlap;
An apparatus for detecting a position of an imaging apparatus. In addition to the first imaging device to the third imaging device, the fourth direction opposite to the first direction is photographed, and a fourth that does not overlap the first image to the third image A position detection device for an image pickup device for detecting a mounting position of a fourth image pickup device for acquiring an image,
The image determination unit inputs the first image to the fourth image captured by the first imaging device to the fourth imaging device, analyzes feature points of the input image, and Determine how the images overlap,
The position identifying unit identifies an attachment position of the first imaging device to the fourth imaging device based on how the images determined by the image determination unit overlap. Position detecting device for the imaging apparatus. A first imaging device that captures a first direction to capture a first image to capture an image of the surroundings of the host vehicle, and a second direction on the left side of the first direction A second imaging device that acquires a second image that partially overlaps the first image, and a third direction on the right side of the first direction, and a portion that overlaps the first image An image pickup apparatus position detection method for detecting an attachment position with a third image pickup apparatus that acquires a third image.
A first step of inputting the first image to the third image captured by the first imaging device to the third imaging device to an image determination unit;
A second step of analyzing the feature points of the images input in the first step and determining how the images overlap in the image determination unit;
A third step of specifying a mounting position of the first imaging device to the third imaging device by a position detection unit based on the overlapping of the images determined in the second step;
The position detection method of the imaging device provided with this.

Images