JPH08125835A - Omniazimuth photographing device and omniazimuth image synthesizer - Google Patents

Abstract

PURPOSE: To provide a panorama image of high resolution and matched view point by dividing the periphery of the photographing device into plural directions, reflecting the images in the respective directions on a planar mirror and photographing those images with correspondent cameras. CONSTITUTION: This device is provided with a camera part 11 for which plural cameras are arranged on a circumference at equal intervals and the optical axes of all the cameras are matched with the normal direction of a circumferential plane where the cameras are arranged, regular polygonal corn shaped reflection mirror 12 joining plural planar mirrors paired with the respective cameras toward the outside, and image synthesizer 13. Then, the camera part and the reflection mirror are arranged so that the central axis of the camera part 11 can be matched with that of the reflection mirror 12, the apex direction of the reflection mirror 12 can be opposite to the photographing direction of the camera part 11 and a virtual image formed by the planar mirrors at the lens centers of cameras comes onto the central axis of the reflection mirror 12, and reflected images photographed on the planar mirrors by the cameras are joined by the image synthesizer 13.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a panoramic image photographing apparatus which photographs all directions.

[0002]

2. Description of the Related Art Conventionally, as a method for taking a panoramic image of a surrounding point at a time from a certain point in space,
For example, MRIU'94IIpp. 151-158.

FIG. 5 is a block diagram of this conventional apparatus.
Reference numeral 1 is a photographing camera, and 52 is a hyperboloidal mirror. Place the hyperboloid mirror vertically downward and the camera vertically upward so that the axis of the hyperboloid and the optical axis of the camera coincide, and the center of the lens of the camera is at the dual position with the focal point of the hyperboloid. .
By capturing the image reflected by the mirror and entering the camera, the side and bottom images are captured at once. The captured image is subjected to correction processing and converted into an image obtained when the viewpoint is placed at the focal point of the hyperboloid.

[0004]

However, in the above-described method, since the scenes in all directions in the surroundings are shown in one image, the resolution is much higher than that in an image normally taken in one direction. It had the problem of falling.

[0005]

SUMMARY OF THE INVENTION The present invention is a camera in which a plurality of cameras are arranged at equal intervals on the circumference and the optical axes of all the cameras are aligned with the normal direction of the circumferential surface on which the cameras are arranged. Department,
A regular polygonal pyramid-shaped reflecting mirror in which a plurality of plane mirrors paired with each camera are joined outwardly, wherein the reflecting mirror is on the lens side of the camera unit and passes through the center of the circumference of the camera unit. The axis directed in the line direction and the central axis of the reflecting mirror are coincident with each other, and the photographing direction of the camera unit and the vertex direction of the reflecting mirror are opposite to each other, and in each pair of the camera and the plane mirror. A plane passing through the optical axis of the camera and the central axis of the reflecting mirror bisects one of the bottom sides of the reflecting mirror,
It is an omnidirectional photographing device in which a virtual image of the lens center of the camera by the plane mirror is arranged on the central axis of the reflecting mirror, and the reflected image of the plane mirror is taken by the camera.

[0006]

By dividing the periphery of the photographing device into a plurality of directions and reflecting the images in the respective directions on the plane mirror and photographing them by the corresponding cameras, the resolution of the images in the respective directions is kept equal to that of a normal camera photographing. .

Further, by arranging the plane mirrors in the shape of a regular polygonal pyramid and matching the virtual images of the lens centers of the cameras at one point in space, an image of the entire circumference with matching viewpoints is obtained.

Further, panoramic image data is obtained by connecting the photographed image data.

Further, by shooting with a plurality of video cameras at the same time, moving images in all directions can be obtained.

Further, a part of the periphery of the photographing device is divided into a plurality of directions, and the images in the respective directions are reflected by a plane mirror and photographed by a corresponding camera to obtain a panoramic image in a necessary range.

[0011]

1 is a block diagram of an omnidirectional photographing apparatus and an omnidirectional image synthesizing apparatus according to a first embodiment of the present invention. In FIG. 1, 11 is a camera unit, 12 is a reflecting mirror, and 13 is an image recording device. The operation will be described below.

The camera section 11 is formed by arranging a plurality of cameras vertically on the circumference. This camera may be any camera as long as it can output a still image. The reflecting mirror 12 is composed of the same number of plane mirrors as the number of cameras of the camera unit 11, and one camera and one plane mirror form one pair. Each camera constituting the camera unit 11 captures the surrounding scene reflected by the corresponding plane mirror.
The captured images are sent to the recording device 13, where all the images are combined and recorded as a panoramic image.

FIG. 2 shows the positional relationship between the reflecting mirror and the camera. (A) is the figure which looked at the whole reflecting mirror which combined a plurality of plane mirrors from the perpendicular upper, and 20 is one plane mirror. (B) shows a positional relationship between a pair of plane mirrors and a camera as viewed from a direction normal to a vertical axis of a reflecting mirror and a plane passing through a lens center of the camera, where 21 is a lens and 22 is a plane mirror. is there. (C) is a view of the reflecting mirror as viewed from the horizontal direction so that one plane mirror comes to the front.

First, (a) will be described. All cameras used for shooting are of the same type, and if the horizontal angle of view is φx, the circumference around a certain vertical axis is θ.
The cameras are arranged so as to be equally divided at an angle θn where n <φx. At this time, the cameras are arranged such that the optical axis of each camera is vertically upward and the vertical direction of the image plane is equal to the normal direction of the circumferential surface on which the cameras are arranged. Let d be the radius of a circle connecting the lens centers of the arranged camera groups and having the vertical axis as the center. Above the group of cameras arranged in this way, a reflector with the same number of plane mirrors as the number of cameras attached to the side of a regular polygonal pyramid is directed vertically downward, and the axis is the vertical axis where the cameras are arranged. The plane mirrors are arranged so as to be equal to each other, and each plane mirror intersects with a plane that passes through the optical axis and the vertical axis of the corresponding camera at a bisector of its vertical angle. Below, how to make this reflector and how to calculate its parameters will be described.

Next, (b) will be described. When viewed from the direction normal to the plane passing through the vertical axis and the optical axis of the camera about the pair of camera and plane mirror, the plane mirror is a straight line. When the point P is the center of the lens and the point Q is the point where the optical axis of the camera passing through the point P intersects the plane mirror, the distance between the points P and Q is h, the tilt angle of the plane mirror with respect to the vertical axis is θm, and the point P is The optical image of the plane mirror is point P ', the vertical angle of view of the camera is φy, and the angle formed by the line segment connecting point P'and point Q with the horizontal direction is θy. The light incident on P is equal to the light incident when there is no plane mirror at a point P ′ which is a virtual image of the point P. From this, the image taken by the camera arranged as shown in (b) has the lens center at the point P ′, the optical axis is equal to P′Q, and the vertical direction of the image plane is the vertical axis and the lens center. It is equivalent to an image taken by a virtual camera arranged parallel to the passing plane. However, an image in which the horizontal direction is reversed can be obtained due to the property of reflection by the mirror.

How much elevation angle is taken around the vertical axis from the horizontal direction is a matter that is decided in advance depending on what kind of image is required. This is equal to the elevation angle θy of the virtual camera having the lens center at the point P ′. Therefore, θy is designated in advance according to the required image state.

Here, the tilt angle θ of the plane mirror with respect to the vertical axis
m is θm = (π / 2−θy) / 2, and the elevation angle θ determined to capture the desired image
Calculate from y.

If the virtual image point P'at the center of the lens obtained for all the sets of cameras and plane mirrors is coincident at one point in space, it is as if the one point is the viewpoint and a part of the image looking around is regarded as each. Can be taken by the camera. As shown in (b), if the point P ′ is on the vertical axis of the circumference of the radius d where the camera is arranged, all the sets can be matched at one point in space due to the symmetry of the device. Since the plane mirror perpendicularly intersects the plane containing the optical axis and the vertical axis of the camera, the point P'is always on this plane. Therefore, if the distance between the optical axis of the camera and the point P'is equal to the radius of the circumference on which the camera is arranged, the point P'is always on the vertical axis. Assuming that the distance between the optical axis of the camera and the point P ′ is d ′, P′Q = PQ = h and <PQP ′ = 2 * θm, and therefore d ′ = h * sin (2 * θm). Here, the distance h between the camera lens center and the plane mirror
If h = d / sin (2 * θm) using the radius d of the circumference, then d ′ = d, and the point P ′ can be brought on the vertical axis.

Therefore, the distance h between the lens center and the plane mirror is determined by the radius d of the circle in which the camera is arranged and the inclination θm of the plane mirror with respect to the vertical axis.

By the way, the vertical angle of view of the camera is φy.
Then, if the length of the plane mirror in the vertical direction is set as lu in the upward direction and ld in the downward direction with reference to the point Q, then lu> h * sin (φy / 2) / sin (θm−φy /
2) ld> h * sin (φy / 2) / sin (θm + φy /
By ensuring the length of 2), it becomes possible to make all the images in the vertical direction captured by the camera into images that are reflected by the plane mirror.

Further, (c) is a view as seen from the normal direction of the straight line connecting the points P'and Q in (b). The horizontal length of the plane mirror is lt = 2d * tan (θm / 2) + 2lu * sin (θ
m) tan (θm / 2) lb = 2d * tan (θm / 2) -2ld * sin (θ
m) tan (θm / 2).

As described above, if the elevation angle of the image to be photographed is determined for a plurality of cameras arranged on a cylinder having a radius of d, the necessary tilt and size of the reflecting mirror and the distance from the camera can be obtained. If a reflecting mirror is used, it is possible to obtain a divided image of the entire circumference with matching viewpoints.

A plurality of cameras of the camera unit 11 simultaneously shoot images, and the data of the taken images are sent to the omnidirectional image synthesizer 13. First of all, the cutout section 14 is
In each image, unnecessary image portions reflected on the adjacent plane mirrors at the left and right edges are removed, and only the mirror image portion at the center is cut out. Next, the inverting unit 15 horizontally inverts each image to convert the mirror image into a normal image. Next, the splicing section 16 turns the image data output from the inverting section 15 to the right when the camera section is viewed from the optical axis direction, as image data in the right direction of the image obtained from a certain camera. The image data obtained from the cameras adjacent to is connected. With one camera as a reference, the image data obtained from all the cameras are connected until the image data obtained from the cameras adjacent to the left side of the camera are connected. The recording unit 17 records the combined data and prepares for output.

As described above, according to the present embodiment, an omnidirectional image having a resolution equivalent to that of normal photographing can be obtained by using the photographing device and the synthesizing device having the above-mentioned configurations.

In this embodiment, the optical axis of the camera has been described as vertically upward. However, the present photographing device may be oriented in any direction, in which case an omnidirectional image in a direction perpendicular to the optical axis of the camera can be obtained. To be

In this embodiment, the camera outputs a still image, but a video camera, a TV camera or the like capable of outputting a moving image may be used, and in this case, the obtained time series images in a plurality of directions are obtained. On the other hand, by connecting images taken at the same time, a moving image of the entire circumference can be obtained.

FIG. 3 is a block diagram of a photographing device and an image synthesizing device according to the second embodiment of the present invention. In FIG. 3, 31 is a camera unit, 32 is a reflecting mirror, and 33 is an image recording device. The operation will be described below.

The camera section 31 comprises a plurality of cameras arranged vertically on an arc. This camera may be any camera as long as it can output a still image. The reflecting mirror 32 is composed of the same number of plane mirrors as the number of cameras of the camera unit 31, and one camera and one plane mirror form one pair. Each camera that constitutes the camera unit 31 captures the surrounding scene reflected by the corresponding plane mirror. The captured images are sent to the recording device 33, where all the images are joined and recorded as a panoramic image.

FIG. 4 shows the positional relationship between the reflecting mirror and the camera. (A) is the figure which looked at the whole reflecting mirror which combined a plurality of plane mirrors from the perpendicular upper direction, and 40 is one plane mirror. (B) shows a positional relationship between a pair of plane mirrors and a camera as seen from a normal direction of a plane passing through the vertical axis of the reflecting mirror and the lens center of the camera, where 41 is a lens and 40 is a plane mirror. is there. (C) is a view of the reflecting mirror as viewed from the horizontal direction so that one plane mirror comes to the front.

First, (a) will be described. The cameras used for shooting are all of the same type, and the angle of view in the horizontal direction is φx. If the direction to be photographed has a horizontal angle of θh, each fan formed by equally dividing the central angle by an angle θn with θn <φx with respect to a fan shape with a central angle of θh and a radius d. Position the cameras so that the center of the camera lens is on the midpoint of the arc of the mold. Where d
Is an appropriate constant. At this time, the cameras are arranged such that the optical axis of each camera is vertically upward and the vertical direction of the image plane is equal to the normal direction of the arc in which the cameras are arranged. The vertical axis that passes through the center of the fan shape where the camera is placed is the center axis of the camera section.

The same number of cameras as the number of congruent flat mirrors having a symmetrical trapezoidal shape are joined to each other in the order of their hypotenuses to form a reflecting mirror having a shape attached to a side surface of a part of a virtual cone. At this time, the end of the reflecting mirror is not closed. Also, the plane mirrors are joined so that the mirror surface of each plane mirror is outside the cone.

The reflecting mirror is arranged so that the apex of the virtual cone is vertically downward and the central axis is equal to the central axis of the camera section. At this time, each plane mirror is a plane that passes through the optical axis and center axis of the corresponding camera and the vertical 2
Arrange them so that they intersect with each other and the plane mirror and the plane mirror intersect vertically.

Below, how to make this reflecting mirror concretely and how to calculate its parameters will be described.

Next, (b) will be described. When viewed from the direction normal to the plane passing through the vertical axis and the optical axis of the camera about the pair of camera and plane mirror, the plane mirror is a straight line. When the point P is the center of the lens and the point Q is the point where the optical axis of the camera passing through the point P intersects the plane mirror, the distance between the points P and Q is h, the tilt angle of the plane mirror with respect to the vertical axis is θm, and the point P is The optical image of the plane mirror is point P ', the vertical angle of view of the camera is φy, and the angle formed by the line segment connecting point P'and point Q with the horizontal direction is θy. The light incident on P is equal to the light incident when there is no plane mirror at a point P ′ which is a virtual image of the point P. From this, the image taken by the camera arranged as shown in (b) has the lens center at the point P ′, the optical axis is equal to P′Q, and the vertical direction of the image plane is the vertical axis and the lens center. It is equivalent to an image taken by a virtual camera arranged parallel to the passing plane. However, an image in which the horizontal direction is reversed can be obtained due to the property of reflection by the mirror.

How much elevation angle is taken around the vertical axis from the horizontal direction is a matter that is determined in advance depending on what kind of image is required. This is equal to the elevation angle θy of the virtual camera having the lens center at the point P ′. Therefore, θy is designated in advance according to the required image state.

Here, the tilt angle θ of the plane mirror with respect to the vertical axis
m is θm = (π / 2−θy) / 2, and the elevation angle θ specified to capture the desired image
Calculate from y.

If the virtual image point P'at the center of the lens obtained for all the sets of cameras and plane mirrors coincides at one point in space, it is as if the one point is the viewpoint and a part of the image looking around is regarded as a part. Can be taken by the camera. As shown in (b), if the point P'is on the center axis of the arc having the radius d in which the camera is arranged, it is possible to match all the sets at one point in space due to the symmetry of the device. Since the plane mirror intersects the plane including the optical axis of the camera and the central axis at right angles, the point P'is always on this plane. Therefore, if the distance between the optical axis of the camera and the point P'is equal to the radius of the arc in which the camera is arranged, the point P'is always on the central axis. Assuming that the distance between the optical axis of the camera and the point P ′ is d ′, P′Q = PQ = h and <PQP ′ = 2 * θm, and therefore d ′ = h * sin (2 * θm). Here, the distance h between the camera lens center and the plane mirror
If h = d / sin (2 * θm) using the radius d of the arc, then d ′ = d and the point P ′ can be brought on the central axis.

Therefore, the distance h between the lens center and the plane mirror is determined from the radius d of the circle in which the camera is arranged and the inclination θm with respect to the center axis of the plane mirror. By the way, assuming that the vertical angle of view of the camera is φy, if the length of the plane mirror in the vertical direction is set to be lu upward and ld downward based on the point Q, then lu> h * sin (φy / 2 ) / Sin (θm-φy /
2) ld> h * sin (φy / 2) / sin (θm + φy /
By ensuring the length of 2), it becomes possible to make all the images in the vertical direction captured by the camera into images that are reflected by the plane mirror.

Further, (c) is a view as seen from the normal direction of the straight line connecting the points P'and Q in (b). The horizontal length of the plane mirror is lt = 2d * tan (θm / 2) + 2lu * sin (θ
m) tan (θm / 2) lb = 2d * tan (θm / 2) -2ld * sin (θ
m) tan (θm / 2).

As described above, if the elevation angle of the image to be photographed is determined for a plurality of cameras arranged on a circular arc of radius d, the necessary tilt and size of the reflecting mirror and the distance from the camera can be obtained. If a reflecting mirror is used, it is possible to obtain a divided image of the surroundings with matching viewpoints.

A plurality of cameras in the camera section 31 simultaneously shoot images, and the data of the taken images are taken by the image synthesizing device 3.
Sent to 3. Here, the clipping unit 34 first removes the unnecessary image portion reflected by the adjacent plane mirrors at the left and right ends of each image, and clips only the mirror image portion at the center. Next, the inverting unit 35 horizontally inverts each image to convert the mirror image into a normal image. Next, the splicing unit 36, as the image data in the right direction of the image data output from the reversing unit 35, is obtained from the camera at the left end when viewed from the central axis, and the image data output from the camera that is adjacent to the camera in the clockwise direction in order. The image data obtained from are connected in the order in which the cameras are arranged. The recording unit 37 records the combined data and prepares for output.

As described above, according to the present embodiment, by using the photographing device and the synthesizing device having the above-mentioned configurations, the same resolution as that of the normal photographing is obtained and the width is as wide as necessary. Images can be obtained.

In the present embodiment, the optical axis of the camera is described as vertically upward, but the present photographing apparatus may be oriented in any direction, in which case an image in a direction perpendicular to the optical axis of the camera can be obtained.

In this embodiment, the camera outputs a still image, but a video camera, a TV camera or the like capable of outputting a moving image may be used. In this case, the obtained time series images in a plurality of directions are obtained. On the other hand, a moving image is obtained by connecting images taken at the same time.

[0045]

According to the present invention, by using a plurality of cameras, it is possible to obtain a panoramic image having a high resolution and matching viewpoints, and it is possible to provide a user with a video having a large amount of information. Its practical effect is great.

Further, according to the present invention, since a panoramic moving image can be obtained by using a plurality of video cameras, its visual effect is great.

Further, according to the present invention, it is possible to selectively photograph only a necessary range around the photographing device.
Its practical effect is great.

[Brief description of drawings]

FIG. 1 is a configuration diagram of an omnidirectional photographing device and an omnidirectional image synthesizing device according to a first embodiment of the present invention.

FIG. 2 is a layout diagram of a reflecting mirror and a camera according to the first embodiment of the present invention.

FIG. 3 is a configuration diagram of a photographing device and an image synthesizing device according to a second embodiment of the present invention.

FIG. 4 is a layout diagram of a reflecting mirror and a camera according to a second embodiment of the present invention.

FIG. 5 is a block diagram of a conventional omnidirectional imaging device.

[Explanation of symbols]

 11 camera section 12 reflecting mirror 13 omnidirectional image synthesizing apparatus 14 clipping section 15 reversing section 16 joining section 17 recording section 20 plane mirror 21 camera lens

Claims (6)

[Claims] 1. A plurality of cameras are arranged at equal intervals on a circumference,
A camera unit in which the optical axes of all the cameras are aligned with the normal direction of the circumferential surface where the cameras are arranged, and a regular polygonal pyramid-shaped reflecting mirror in which a plurality of plane mirrors paired with each camera are joined outward The reflecting mirror is on the lens side of the camera unit, and an axis directed through a center of a circumference of the camera unit and directed in a normal direction is coincident with a central axis of the reflecting mirror. Direction and the vertex direction of the reflecting mirror are opposite to each other, and in each pair of the camera and the plane mirror forming a pair, the plane passing through the optical axis of the camera and the central axis of the reflecting mirror is the base of the reflecting mirror. One of the two is bisected, the virtual image of the lens center of the camera by the plane mirror is on the central axis of the reflecting mirror,
An omnidirectional photographing device for photographing a reflection image of the plane mirror with the camera. 2. A cutout part for cutting out data of a mirror image part shown in the central part from a plurality of image data taken by the omnidirectional image pickup device according to claim 1, and data of a mirror image part cut out by the cutout part. The reverse part that generates image data in which the horizontal direction of the screen is reversed, the joint part that joins the reverse image data generated by the reverse part from the images captured by the adjacent cameras, and the joined data is recorded. An omnidirectional image synthesizing device including a recording unit. 3. A camera section in which a plurality of video cameras are arranged at equal intervals on a circumference, and the optical axes of all the video cameras are aligned with the normal direction of the circumferential surface on which the video cameras are arranged. The omnidirectional photographing device according to claim 1, wherein the omnidirectional photographing device is replaced with. 4. A cutout part for cutting out data of a mirror image part shown in the central part from a plurality of time-series image data taken by the omnidirectional image pickup device according to claim 3, and a mirror image part cut out by the cutout part. The image is inverted from the image data in the horizontal direction to generate image data, and the image data captured by adjacent cameras at the same time is combined with the inverted image data generated by the image inversion unit. An omnidirectional image synthesizing device including a recording unit that records data. 5. A camera unit in which a plurality of cameras are arranged at equal intervals on an arc forming a part of the circumference, and the optical axes of all the cameras are aligned with the normal direction of the circumferential surface on which the cameras are arranged. A plurality of plane mirrors that are paired with individual cameras are joined outwardly, and a reflecting mirror is provided which has a shape of a side surface of a regular polygonal pyramid, and the reflecting mirror is on the lens side of the camera unit. An axis directed through a center of an arc of the camera unit and directed in a normal direction coincides with a central axis of the reflecting mirror, a mirror surface of the reflecting mirror faces the camera unit, and a pair of the camera and the camera In each set of plane mirrors, a plane that is the optical axis of the camera and the central axis of the arc of the camera section intersects perpendicularly with the plane mirror, and the virtual image of the lens center of the camera by the plane mirror is the center of the arc of the camera section. On-axis, a reflection image of the plane mirror is taken by the camera. Imaging apparatus for photographing. 6. A cutout unit for cutting out data of a mirror image portion shown in the central portion from a plurality of image data taken by the image pickup apparatus according to claim 5, and a screen from the data of the mirror image portion cut out by the cutout unit. An inversion unit that generates image data in which the horizontal direction is inverted, a joining unit that joins the inverted image data generated by the inversion unit from images captured by adjacent cameras, and a recording unit that records the joined data. Image synthesizing device.