JP3126157B2 - Image information processing device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image information processing apparatus.

[0002]

SUMMARY OF THE INVENTION According to the present invention, an image such as a panoramic image is stored in a storage means, and a process based on a predetermined algorithm is performed on the image to obtain an image having a perspective equivalent to, for example, a real photograph. It is possible,
Furthermore, panning in a horizontal direction of 360 degrees is enabled.

[0003]

2. Description of the Related Art As a conventional technique, there is an apparatus shown in FIG. 7 which uses an image captured by a wide-angle flat image pickup system as an original image. In FIG. 7, the original image is a frame memory 1
The portion of the area 2 formed by x corresponding to the pan angle θ, y corresponding to the tilt angle φ, and m corresponding to the zoom magnification z is cut out therefrom and input to the perspective conversion unit 3. In the perspective conversion unit 3, for example, as shown in FIG. 8, for the original image of (a), (b), (c),
The target image is obtained by performing the geometric transformation as shown in (d) and (e). (B) is x <0, (c) is x>
0, (d) shows the output image of the perspective conversion unit 3 at y <0, and (e) shows the output image at y> 0.

[0004]

In the above-mentioned prior art, there is a problem in that a perspective equivalent to that of a real photograph is not given, which causes a sense of incongruity. For example, as shown in FIG. 6, when a square in which a lattice pattern of (a) is drawn is considered as an original image, the four points of the four corners match as shown in (b) as an image after perspective conversion.
An image with an incorrect scaling ratio inside the square is obtained.
In addition, since the image captured by the planar imaging system is used as the original image, panning and
There is a problem that the image disappears when tilting is attempted.

An object of the present invention is to provide an image information processing apparatus which has solved the above-mentioned problems.

[0006]

In order to achieve the above object, the present invention provides a storage means for storing panoramic image information and a method for projecting the panoramic image information in the storage means onto a first cylindrical peripheral surface. first algorithm by mapping from the panoramic image on a first cylindrical peripheral surface to the image plane corresponding to the pan constituting the image hazy perspectives, and the
Perspective by the first algorithm on the image plane
And mapping active with took an image on each corresponding image plane tilt and zoom from the second panorama image on the cylindrical peripheral surface when projected into a second cylindrical peripheral surface perpendicular to the first cylindrical Means for subjecting the panoramic image information in the storage means to perspective conversion based on a second algorithm for forming a perspective-applied image.

[0007]

According to the present invention, for example, an image with the same perspective as an image taken by an actual camera can be obtained, and panning in a horizontal direction of 360 degrees can be performed.

[0008]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS First, an example of a method for capturing a panoramic image serving as an original image will be described with reference to FIG. A vertical line sensor camera 6 is attached to a pan axis 5 to which an angle detector 4 is attached, and can be rotated by 360 degrees. Reference numeral 9 denotes a subject of the line sensor camera 6. The line sensor camera 6 has a lens 6A and a line sensor 6B provided on the image forming surface thereof. The output from the line sensor 6B is A / D converted by an A / D converter 7 and Writing is performed for one line. At this time, the writing position to the frame memory 8 is controlled based on a detection signal indicating the rotation angle of the pan shaft 5 from the angle detector 4 so that writing is performed at a position corresponding to the pan angle. With this device,
For example, if the pan axis 5 is rotated 360 degrees, the horizontal direction 360
An image corresponding to the degree is obtained on the frame memory 8. Such an image captured over a wide range is called a panoramic image, and is an original image in the present invention. This panoramic image can also be directly generated using computer graphics and written on the frame memory 8.

Next, a description will be given of a geometric conversion algorithm for converting a panoramic image into a target image according to the present invention.
As shown in FIG. 2A, the panoramic image has a cylindrical surface 11 with a radius of 1 around a vertical axis 10,
Landscape outside the cylinder, centered on point O above (example: subject 9)
Is projected onto the peripheral surface (hereinafter, referred to as a cylindrical surface) 11 of the cylinder, and can be considered as an image formed on the cylindrical surface 11. Now, as shown in FIG. 2B, consider an imaging plane 12 panned about an axis 10 by an angle θ. Here, assuming that a point at which the straight line connecting the point P and the point O on the cylindrical surface 11 intersects the plane 12 is a point Q, the pixel value of the point Q is set to the pixel value of the point P, and the column surface
11 is obtained by panning the panorama image on the image 11 at an angle θ. Next, in FIG. 2B, a point where a tangent between the cylindrical surface 11 and the plane 12 intersects a horizontal plane passing through the point O is defined as R, and an axis 13 orthogonal to the OR on the horizontal plane including the point O is considered. In FIG. 2C, a cylindrical surface 14 having a radius of 1 around the axis 13 is considered, and a point at which the straight line connecting a certain point S and a point O on the plane 12 intersects with the cylindrical surface 14 is defined as a point T. Is obtained as the pixel value of the point S, a vertical panoramic image of an image panned at an angle θ on the plane 12 on the cylindrical surface 14 is obtained. Next, as shown in FIG. 2D, on a line where the vertical plane passing through the point O and the cylindrical surface 14 intersect, the line is orthogonal to a straight line connecting the point U and the point O corresponding to the angle φ, and Considering a plane 15 whose distance is z, this is an imaging plane at a zoom magnification z that is tilted about the axis 13 by an angle φ. Therefore, when the point at which the straight line and the plane 15 connecting the V and the point O point on the cylindrical surface 14 intersects the point W, by the pixel value of the point W and the pixel value of the point V, on the plane 15, the plane 1
2. Chill the image panned at angle θ above by angle φ
To obtain an image zoomed at the zoom magnification z . FIG.
(C) shows an example, and an image equivalent to a perspective in an actual camera is obtained.

The above algorithm is formulated. As shown in FIG. 3A, a panoramic image serving as an original image is p-
An x-y coordinate is set on an output image obtained when a camera operation corresponding to a pan angle θ, a tilt angle φ, and a zoom magnification z is performed as shown in FIG. Is set, the coordinate conversion equation between the panoramic image and the output image is as shown in the following equations (1) and (2).

[0011]

(Equation 1)

[0012]

(Equation 2)

Two embodiments will be described as hardware for realizing the geometric transformation given by Equations (1) and (2). First, the embodiment of FIG. 4 will be described. The panoramic image serving as the original image is stored in the frame memory 8. The desired pan angle θ, tilt angle φ, and zoom magnification z are input to the address calculation unit 16, where the address of the pixel to be extracted from the panoramic image is calculated according to Expressions 1 and 2, and the result is subjected to address control. It is input to the section 17 and controls the read address of the frame memory 8. The digital data output from the frame memory 8 according to the read address is D / A converted by the D / A converter 18 and output as a video signal.

Next, the embodiment shown in FIG. 5 will be described. This focuses on the form of Equations (1) and (2), and separates horizontal cropping processing by panning, geometric transformation processing dependent on tilt, and enlargement / reduction processing dependent on zoom magnification. By transforming Equations (1) and (2), horizontal cutting by panning, geometric transformation depending on tilt, and scaling depending on zoom magnification are performed as in the following Equations (3), (4) and (5). Become. However, p-
For the q coordinate, u is the coordinate of the image processed by panning.
The coordinates of the image subjected to the tilt process with respect to the uv coordinates are referred to as st coordinates, and the coordinates of the final output image obtained by performing the zoom process on the uv coordinates are referred to as xy coordinates. I do.

[0015]

(Equation 3)

[0016]

(Equation 4)

[0017]

(Equation 5)

A panoramic image serving as an original image is stored in a frame memory 8. The desired pan angle θ is input to the horizontal cutout address calculation unit 19, where the readout address from the frame memory 8 is calculated by Expression 3, and the result is input to the address control unit 20, and based on this input, The address control unit 20 controls a read address of the frame memory 8. The data output from the frame memory 8 according to the read address is input to the geometric conversion unit 21. On the other hand, the desired tilt angle φ is input to the geometric transformation calculating unit 22, where a geometric transformation formula based on the desired tilt angle φ is calculated by Expression 4, and the geometric transformation unit 21 calculates A process corresponding to tilt is performed on the input data, and the processed data is input to the scaling unit 23. On the other hand, the desired zoom magnification z is input to the enlargement / reduction calculation unit 24, where a reduction conversion formula based on the desired zoom magnification is calculated by Expression 5, and
The enlargement / reduction unit 23 performs a process corresponding to the zoom on the input data according to the result, and the data after the process is D / A converted by the D / A conversion unit 25 and output as a video signal.

[0019]

According to the present invention, correct reproduction of a perspective in an output image is enabled, and a horizontal 360
Degree of panning is possible.

[Brief description of the drawings]

FIG. 1 is a diagram illustrating an example of a panorama image capturing device according to each embodiment of the present invention.

2A and 2B are diagrams illustrating a conversion algorithm from a panoramic image to a target image, wherein FIG. 2A illustrates a panoramic image, and FIG. 2B illustrates an image on a plane imaging plane rotated by a pan angle θ. (C) is an explanatory diagram of a panoramic image in the vertical direction, and (d) is an explanatory diagram of an image obtained by rotating the camera by a tilt angle φ and performing zooming at a zoom magnification z.

FIG. 3 is a diagram illustrating coordinates of a panoramic image and an output image.

FIG. 4 is a block diagram of a first embodiment of the present invention.

FIG. 5 is a block diagram of a second embodiment of the present invention.

FIG. 6 is a diagram for explaining a difference between a conventional example and the present invention in perspective conversion.

FIG. 7 is a block diagram showing a schematic configuration of a conventional technique.

FIG. 8 is a diagram showing the concept of perspective processing in the related art.

[Explanation of symbols]

 8 Frame memory 16 Address calculation unit 17 Address control unit 18 D / A conversion unit

Claims (1)

(57) [Claims] An accumulating means for accumulating panoramic image information; and, when projecting panoramic image information in the accumulating means onto a first cylindrical peripheral surface, the panoramic image on the first cylindrical peripheral surface corresponds to panning. first algorithm by mapping the image plane constituting the image hazy perspectives to, and before
Parsing by the first algorithm on the image plane
And mapping took pictures of Pekutibu to each corresponding image plane tilt and zoom from the second panorama image on the cylindrical peripheral surface when projected into a second cylindrical peripheral surface perpendicular to the first cylindrical Means for performing a perspective conversion of the panoramic image information in the storage means on the basis of a second algorithm for forming a perspective-applied image.