JP2007048071A - Image processor and image processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To prevent that one three-dimensional object is displayed as a plurality of three-dimensional objects when creating one image by applying viewpoint conversion processing to a plurality of images imaged by a plurality of cameras and composing them. <P>SOLUTION: This image processor converts images imaged by the plurality of cameras 1, 2 into viewpoint conversion images in a view from a viewpoint different from viewpoints of the cameras 1, 2, composing the plurality of converted viewpoint conversion images to create one composite image. In the image processor, the three-dimensional object extending from one start point to another direction is detected on the composite image, and redrawn as one three-dimensional object. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an apparatus and a method for creating a single composite image by performing viewpoint conversion processing on images picked up by a plurality of image pickup means.

  2. Description of the Related Art Conventionally, there is known an apparatus that creates one image by converting two images captured at different positions into a viewpoint that is different from the captured viewpoint (see Patent Document 1).

JP 2001-114047 A

  However, the conventional technique has a problem that a three-dimensional object such as a pole-shaped three-dimensional object is not displayed properly, such as being displayed twice on a composite image.

(1) An image processing apparatus according to the present invention converts images captured by a plurality of imaging units into viewpoint converted images viewed from a viewpoint different from the viewpoint of the imaging unit, and converts the converted plurality of viewpoint converted images. A composite image is created to create a single composite image, and a three-dimensional object appearing in a plurality of directions extending from one start point on the composite image is detected and redrawn as a single solid object. To do.
(2) The image processing method according to the present invention converts viewpoints of images captured by a plurality of imaging means into images viewed from a viewpoint different from the viewpoints of the plurality of imaging means (hereinafter referred to as viewpoint converted images). And then creating one composite image, detecting a three-dimensional object appearing in a plurality of directions from one start point on the composite image, and redrawing as one solid object To do.

  According to the image processing device and the image processing method of the present invention, a plurality of viewpoint-converted images are combined to create one composite image, and a three-dimensional object extending in one or more directions from one start point is detected on the composite image. Then, since it is redrawn as one solid object, it is possible to prevent one solid object from being displayed as a plurality of solid objects on the composite image.

  FIG. 1 is a diagram illustrating a configuration of an image processing apparatus according to an embodiment. An image processing apparatus according to an embodiment is used by being mounted on a vehicle, and includes a camera 1, a camera 2, a processing apparatus 3, and a display 4. The camera 1 and the camera 2 are, for example, CCD cameras, and are provided at different positions on the vehicle to image the surroundings of the vehicle.

  The processing device 3 includes a viewpoint conversion unit 31, an image synthesis unit 32, a three-dimensional object detection unit 33, and a three-dimensional object drawing unit 34 in terms of processing functions performed internally. The viewpoint conversion unit 31 performs a coordinate conversion process for converting the image captured by the camera 1 and the image captured by the camera 2 into a viewpoint conversion image captured from an arbitrary position. Specifically, it is assumed that a planar road surface is reflected in all the vehicle surrounding images, and a viewpoint conversion image obtained by observing the road surface from an arbitrary position (for example, directly above the vehicle) is generated. Since the viewpoint conversion process is a known technique, a detailed description of the viewpoint conversion process is omitted here.

  The image composition unit 32 composes the two images whose viewpoints have been converted, and creates a composite image. Here, the two images are combined so that the same object appearing on each image overlaps. The three-dimensional object detection unit 33 detects a three-dimensional object appearing on the composite image created by the image composition unit 32, particularly a three-dimensional object appearing on the composite image so as to extend from one starting point to another direction. To do. The three-dimensional object drawing unit 34 redraws the three-dimensional object detected by the three-dimensional object detection unit 33 as one solid object on the composite image created by the image composition unit 32. On the display 4, a composite image in which a three-dimensional object is redrawn is displayed.

  FIG. 2 is a flowchart showing the contents of processing performed by the image processing apparatus according to the embodiment. When the vehicle is activated, the processing device 3 starts the process of step S10. In step S10, images captured by camera 1 and camera 2 are acquired, and the process proceeds to step S20. In step S20, viewpoint conversion processing is performed by the viewpoint conversion unit 31 on the two images acquired in step S10. Thereby, the images captured by the camera 1 and the camera 2 are converted into images captured from the same viewpoint.

  In step S30 following step S20, the two images converted by the viewpoint conversion unit 31 are combined by the image combining unit 32 to create a combined image. When the composite image is created, the process proceeds to step S40. In step S <b> 40, the three-dimensional object detection unit 33 detects a three-dimensional object appearing on the composite image so as to extend from one starting point in another direction. First, the three-dimensional object detection unit 33 compares the density (luminance) in the same pixel of the viewpoint converted image of the image captured by the camera 1 and the viewpoint converted image of the image captured by the camera 2. If the difference in density in the same pixel is less than a predetermined value, it is determined that the pixel is an area where a road surface is imaged. If the density difference is greater than or equal to a predetermined value, the pixel is an area where a three-dimensional object is imaged. Judge that there is.

  Next, a process of detecting the edge of the three-dimensional object is performed on the region where it is determined that the three-dimensional object is imaged. FIG. 3 is a diagram illustrating the edges of the three-dimensional object 50 captured by the camera 1 and the edges of the three-dimensional object 50 captured by the camera 2 on the composite image. Of the edges of the three-dimensional object 50 imaged by the camera 1, as shown in FIG. 3, the ones continuously extending in one direction are set as vectors a1 ′ and a2 ′ (a1 ′ and a2 ′ are respectively Represents a vector symbol). The point where the distance between the two vectors a1 ′ and a2 ′ is the shortest is the start point 51 of both vectors, and when both vectors have a divergence angle with respect to the start point, they are composed of two edges. The three-dimensional object is determined to be a pole-shaped three-dimensional object extending in the vertical direction.

  Further, as shown in FIG. 3, among the edges of the three-dimensional object 50 imaged by the camera 2, those continuously extending in one direction are set as vectors b 1 ′ and b 2 ′ (b 1 ′ and b 2 ′ are , Each representing a vector symbol). The point where the distance between the two vectors b1 ′ and b2 ′ is the shortest is the start point 51 of both vectors, and when both vectors have a divergence angle based on the start point, the two vectors b1 ′ and b2 ′ are composed of two edges. The three-dimensional object is determined to be a pole-shaped three-dimensional object.

  Here, when the starting points of the vectors a1 ′ and a2 ′ coincide with the starting points of the vectors b1 ′ and b2 ′, the three-dimensional object constituted by the vectors a1 ′ and a2 ′ and the vectors b1 ′ and b2 ′. Are determined to be the same three-dimensional object. That is, the three-dimensional objects captured by the camera 1 and the camera 2 are reflected as different three-dimensional objects on the composite image.

In step S40 of the flowchart shown in FIG. 2, when the detection processing of the three-dimensional object extending from one starting point in another direction is performed, the process proceeds to step S50. In step S50, the position for redrawing the three-dimensional object detected in step S40 is specified on the composite image. Here, the region surrounded by the vector c1 ′ represented by the following equation (1) and the vector c2 ′ represented by the following equation (2) is set as the position of the three-dimensional object.
c1 ′ = (a1 ′ + b1 ′) / 2 (1)
c2 ′ = (a2 ′ + b2 ′) / 2 (2)

In step S60 following step S50, the three-dimensional object detected in step S40 is redrawn on the composite image created in step S30. Here, when the vectors a3 ′, b3 ′, and c3 ′ are respectively expressed by the following equations (3) to (5), a region captured by the vectors c1 ′ and c3 ′ is a vector captured by the camera 1. An image obtained by reducing the image of the area surrounded by a1 ′ and a3 ′ by (| c3 ′ | / | a3 ′ |) is displayed. Further, in the area surrounded by the vectors c2 ′ and c3 ′, the video of the area surrounded by the vectors b2 ′ and b3 ′ captured by the camera 2 is reduced by (| c3 ′ | / | b3 ′ |). Display what you did.
a3 ′ = (a1 ′ + a2 ′) / 2 (3)
b3 ′ = (b1 ′ + b2 ′) / 2 (4)
c3 ′ = (c1 ′ + c2 ′) / 2 (5)

  Thus, by synthesizing a part of the image of the three-dimensional object 50 captured by the camera 1 and a part of the image of the three-dimensional object 50 captured by the camera 2, the image of the three-dimensional object 50 is drawn on the composite image. At the time of correction, the size of the image to be synthesized is determined based on the size of the image of the three-dimensional object 50 captured by both the cameras 1 and 2, so the image of the three-dimensional object 50 captured by the cameras 1 and 2. Even when the sizes of the three-dimensional objects are different, it is possible to prevent the shape of the three-dimensional object drawn on the composite image from becoming discontinuous.

  Further, on the composite image, the road image captured by the camera 2 is displayed in the area surrounded by the vectors a1 ′ and a2 ′, and the camera 1 is displayed in the area surrounded by the vectors b1 ′ and b2 ′. The image of the road surface imaged at is displayed. This makes it possible to appropriately represent the portion where the video disappears when the solid object that has been projected to extend in another direction is redrawn as one solid object.

  According to the image processing apparatus in the embodiment, images respectively captured by the plurality of cameras 1 and 2 are converted into viewpoint conversion images viewed from a viewpoint different from the viewpoints of the cameras 1 and 2, and the converted plurality of images are converted. In the image processing apparatus that synthesizes the viewpoint-converted images to create one composite image, the solid image is detected on the composite image so as to extend in a plurality of directions from one start point, and one solid object is detected. Redraw as. Thereby, it is possible to prevent one solid object from appearing as a plurality of solid objects on the composite image.

  Further, according to the image processing apparatus in the embodiment, the position of the three-dimensional object to be redrawn on the composite image is determined based on the position of the image when the same three-dimensional object is captured by the plurality of cameras 1 and 2. Therefore, the three-dimensional object can be redrawn at an appropriate position on the composite image.

  According to the image processing apparatus in the embodiment, a part of an image when a plurality of cameras 1 and 2 capture a three-dimensional object that is projected from a single start point in a plurality of directions is combined. Thus, one solid object is redrawn. At this time, the size of the image to be combined when one solid object is redrawn is determined based on the size of the image taken by each of the plurality of cameras 1 and 2. Thereby, when the magnitude | size when a solid object is imaged with a some camera differs, it can prevent that the shape of the solid object drawn on a synthesized image becomes discontinuous.

  Furthermore, according to the image processing apparatus in one embodiment, a plurality of cameras 1 and 2 capture an image of a region in which a three-dimensional object is projected so as to extend in a plurality of directions from one starting point on a composite image. Since the image is redrawn based on the rendered image, the portion where the video disappears by redrawing the image on one solid object can be appropriately expressed.

  The present invention is not limited to the embodiment described above. For example, in the above-described embodiment, the example in which the image processing apparatus is mounted on the vehicle and the surroundings of the vehicle are imaged by the cameras 1 and 2 has been described. However, the present invention can be applied to other than the vehicle. Further, the number of cameras is not limited to two, and three or more cameras may be provided. In this case, if a plurality of three-dimensional objects extending in the other direction from the same starting point are reflected on the composite image, the plurality of three-dimensional objects may be redrawn as one solid object.

  Even when the three-dimensional object 50 is a moving object, the three-dimensional object 50 can be redrawn on the composite image. A drawing method when the three-dimensional object 50 is a moving object will be described with reference to FIG. FIG. 4 is a diagram illustrating a composite image converted into a viewpoint directly above the vehicle. In FIG. 4, the imaging range of the camera 1 is indicated by boundary lines 11 and 12, and the imaging range of the camera 2 is indicated by boundary lines 21 and 22. The intersection of the boundary line 11 and the boundary line 21 is set to 40.

When the angle formed between the line 41 connecting the intersection 40 and the three-dimensional object 50 and the boundary line 11 is θ2, and the angle formed between the line 41 connecting the intersection 40 and the three-dimensional object 50 and the boundary line 21 is θ1, the position of the three-dimensional object 50 is determined. A vector c3 ′ for drawing is expressed by the following equation (6).
c3 ′ = (θ2 · a3 ′ + θ1 · b3 ′) / (θ1 + θ2) (6)
According to equation (6), when θ1 = 0, c3 ′ = a3 ′, and when θ2 = 0, c3 ′ = b3 ′.

  By expressing the vector c3 ′ for drawing the position of the moving object by the equation (6), the movement of the three-dimensional object in the vicinity of the boundary line 11 of the imaging range of the camera 1 or the boundary line 21 of the imaging range of the camera 2 is smoothed. Can be expressed.

  In the above-described embodiment, the three-dimensional object on the captured image is detected by comparing the density at the same pixel of the images captured by the cameras 1 and 2, but the colors at the same pixel of both images are compared. A three-dimensional object can also be detected by other methods.

  The correspondence between the constituent elements of the claims and the constituent elements of the embodiment is as follows. That is, the cameras 1 and 2 are imaging means, the viewpoint conversion unit 31 is viewpoint conversion means, the image composition unit 32 is image synthesis means, the three-dimensional object detection unit 33 is three-dimensional object detection means, and the three-dimensional object drawing unit 34 is three-dimensional. Each object drawing unit is configured. In addition, the above description is an example to the last, and when interpreting invention, it is not limited to the correspondence of the component of said embodiment and the component of this invention at all.

The figure which shows the structure of the image processing apparatus in one embodiment The flowchart which shows the processing content performed by the image processing apparatus in one embodiment The figure which shows the edge detected when a pole-shaped solid object is imaged with two cameras The figure which shows the composite image converted into the viewpoint right above the vehicle

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Camera, 2 ... Camera, 3 ... Processing apparatus, 4 ... Display, 31 ... Viewpoint conversion part, 32 ... Image composition part, 33 ... Three-dimensional object detection part, 34 ... Three-dimensional object drawing part

Claims (6)

A plurality of imaging means;
Viewpoint conversion means for converting images respectively captured by the plurality of imaging means into images viewed from viewpoints different from viewpoints of the plurality of imaging means (hereinafter referred to as viewpoint conversion images);
Image synthesizing means for synthesizing a plurality of viewpoint converted images converted by the viewpoint converting means to create one synthesized image;
Three-dimensional object detection means for detecting a three-dimensional object appearing in a plurality of directions from one starting point on the composite image created by the image composition means;
An image processing apparatus comprising: a three-dimensional object drawing unit that redraws a three-dimensional object detected by the three-dimensional object detection unit as one solid object. The image processing apparatus according to claim 1.
The three-dimensional object drawing means specifies the position of the three-dimensional object to be redrawn based on the position of the three-dimensional object appearing to extend in a plurality of directions with one start point as a reference on the composite image. A featured image processing apparatus. The image processing apparatus according to claim 1 or 2,
The three-dimensional object drawing means redraws one solid object by synthesizing a part of an image when the three-dimensional object detected by the three-dimensional object detection means is respectively captured by the plurality of imaging means. A featured image processing apparatus. The image processing apparatus according to claim 3.
The three-dimensional object drawing unit is configured to redraw the one three-dimensional object based on the size of an image when the three-dimensional object detected by the three-dimensional object detection unit is captured by the plurality of imaging units. An image processing apparatus that determines a size of an image to be processed. The image processing apparatus according to any one of claims 1 to 4,
The three-dimensional object drawing unit is configured to, based on the images captured by the plurality of imaging units, images of regions in which the three-dimensional object is projected so as to extend in a plurality of directions from the one start point on the composite image. An image processing apparatus characterized by redrawing. Each of the images picked up by the plurality of image pickup means is subjected to viewpoint conversion into an image viewed from a viewpoint different from the viewpoint of the plurality of image pickup means (hereinafter referred to as viewpoint conversion image),
Create a composite image by combining multiple viewpoint-converted images that have undergone viewpoint conversion,
An image processing method, comprising: detecting a three-dimensional object appearing in a plurality of directions extending from one starting point on the composite image, and redrawing the three-dimensional object.

Images