JP4254667B2 - Wide viewing angle imaging device - Google Patents

Description

  The present invention relates to an apparatus that performs imaging with a wide viewing angle. In particular, the present invention relates to a wide viewing angle video imaging apparatus that automatically changes the position and orientation of a camera in accordance with a shooting range.

  Wide-angle images can be captured by combining a convex mirror and a single camera, by combining multiple plane mirrors and multiple cameras, by synchronizing multiple cameras side by side, etc. There is.

  The method of shooting using a combination of a convex mirror and one camera can obtain wide-angle and omnidirectional images from the same optical center. It is difficult to use as a highly realistic content such as display on a large screen.

  The method of shooting with a combination of multiple plane mirrors and multiple cameras can obtain wide viewing angle and omnidirectional images at the same time from the same optical center by adjusting the mirror angle and camera position angle. Although it is possible and suitable for use in highly realistic content, it is necessary to work with the equipment, and the precision of the equipment is required because it greatly affects the image where the work accuracy of the equipment can be obtained. Since fine adjustments are required, preparation for shooting is difficult.

  In the method of shooting by synchronizing a plurality of cameras side by side, the optical center of each camera is different due to the physical restrictions of the cameras, and therefore, in the stitching of images for creating a wide viewing angle image, an inconsistency occurs in the joint portion. In wide-angle image shooting with multiple cameras arranged side by side, the method of shooting with the camera lens facing outward is generally used, but the optical center of each camera differs greatly, so the joints are inconsistent. There is a drawback that stands out. In particular, a subject that is close to the camera has a problem that a blind spot occurs, and a problem that an overlap between adjacent images is reduced. Therefore, the subject is not suitable for photographing at a relatively close distance to the subject.

  Examples of apparatuses that capture wide-angle images with a plurality of cameras include the following apparatuses.

  Japanese Patent Laying-Open No. 2004-61808 (Patent Document 1) discloses an apparatus that shoots an omnidirectional image by arranging cameras with a lens facing outward in a cylindrical shape.

  The following apparatus is mentioned as an apparatus provided with the camera adjustment mechanism in the imaging | photography apparatus by a some camera.

  Japanese Patent Laid-Open No. 2002-57938 (Patent Document 2) discloses an apparatus that automatically changes the camera posture in accordance with a change in photographing magnification in a configuration in which a plurality of cameras are arranged with the lens facing outward. Yes. In this device, even if there is a change in shooting magnification even in the configuration of multiple cameras, the effect of shooting with one camera without being aware of the joint is obtained by automatically changing the shooting range. It is possible.

JP 2004-61808 A JP 2002-57938 A

  In the technique disclosed in Patent Document 1, since the lens faces outward, the optical center of each camera is greatly shifted, and inconsistency occurs at the joint portion when creating a panoramic image. In addition, since the camera position and orientation and the camera parameters are uniquely determined, it is difficult to cope with a change in the required viewing angle.

  With the apparatus disclosed in Patent Document 2, even if there is a change in the shooting magnification even in the configuration of a plurality of cameras, a single camera can be used without being aware of the joints by automatically changing the shooting range. It is possible to obtain the effect as if it was photographed. However, since the optical centers of the cameras are far apart, it is possible to create a seamless panoramic image when the background image is in the distance or when the distance from the camera of the subject to be photographed is constant. However, when there are a plurality of subjects having different depths at the joint portion of the video, a large mismatch occurs at the joint portion, so that seamless image connection is difficult.

  As described above, in the conventional wide viewing angle image camera using a plurality of cameras, the optical center of all the cameras is close, and the camera configuration does not automatically change according to the required change of the photographing range. .

  The present invention was devised in view of the above problems, and its purpose is to automatically determine the camera configuration and adjust the position and orientation of the camera when an arbitrary shooting range is input. is there.

  Therefore, in the present invention, in a wide viewing angle video imaging apparatus configured by a plurality of cameras provided with a focal length control device and a plurality of rotational movement devices equipped with each camera, a total field angle parameter indicating an imaging range is adjacent. Camera angle-of-view calculating means for calculating the angle of view and angle of each camera from the input total angle-of-view parameter and the input overlap ratio, and the calculated camera A focal length calculation unit that calculates a focal length of the camera from the angle of view of the camera, a camera rotation amount calculation unit that calculates a rotation amount of each camera from the calculated focal length and a camera posture initial value, and the calculated camera rotation By providing camera movement amount calculation means that brings the optical center of each camera closer to the camera's initial position from the camera's initial position Capable of automatically controlling the energization to provide a wide viewing angle imaging apparatus.

  As an effect of the present invention, by adjusting the camera posture in accordance with the shooting range, the required shooting range and overlap ratio are satisfied, and the joining of adjacent camera images is facilitated.

  Embodiments of the present invention will be described in detail below with reference to the drawings.

  FIG. 1 is a diagram showing the configuration of the apparatus of this embodiment. The configuration of five cameras will be described as an example.

  In photographing a wide viewing angle image, first, five cameras (10-a, 10-b, 10-c, 10-d, 10-e) are arranged in a fan shape so that the lenses face inward. All cameras shall be equipped with focal length adjusting devices (20-a, 20-b, 20-c, 20-d, 20-e). Each camera shall be installed on a rotational movement device (30-a, 30-b, 30-c, 30-d, 30-e), and the rotational movement device shall be the horizontal rotation of the camera and the optical axis direction. Control movement to.

  When installing the cameras side by side, the rotary movement device (30-) equipped with the cameras (10-a, 10-b, 10-c, 10-d, 10-e) so that the horizontal axis of each camera is the same. a, 30-b, 30-c, 30-d, 30-e) may be installed on the base plate (40).

  FIG. 2 is a diagram showing an internal configuration in the present embodiment.

  The total angle-of-view parameter (220) that indicates the range to be shot and the overlap ratio (230) that indicates the overlapping part of the images shot by the adjacent camera (10) are input parameters by manual input or reading from the data. Then, the camera angle-of-view calculation unit (110) calculates the posture parameter and the camera angle of view of the camera (10) that is most effective for shooting, which maximizes the shooting range of each camera (10).

  From the calculated camera angle of view, the focal length calculation unit (120) calculates the focal length of each camera, and based on the calculated focal length, the focal length control device (20) uses each camera (10 ) Automatically adjusts the focal length.

  The installation angle of each camera is determined based on the calculated camera angle of view and the overlap ratio (230) of each camera (10). Therefore, the camera rotation amount calculation unit (130) calculates how much each camera is rotated from each camera posture initial value (210) and the determined camera installation angle. Also, the camera movement amount calculation unit (140) calculates how much each camera is moved so that the optical center of each camera is closest to each other based on each camera posture initial value (210) and the calculated camera rotation amount. Based on the calculated camera rotation amount and camera movement amount, the rotational movement device (30) adjusts the horizontal angle of each camera (10) and the position of each camera (10) in the optical axis direction. Make adjustments automatically.

  FIG. 3 is a diagram showing a system configuration of the present embodiment.

  This system includes a CPU (60) that performs calculation and control based on a program, a main storage device (100) such as a memory that stores data and programs, and an auxiliary storage device such as a hard disk, which is included in the server PC (50) (200) included.

  Further, it includes an input device (70) such as a keyboard for inputting various data, a rotational movement device (30) for performing rotation control and movement control of the camera (10), and a bus (80) for coupling them together. A camera (10) is mounted on the rotary movement device (30), and each camera (10) is provided with a focal length control device (20).

  The auxiliary storage device (200) is adjacent to the camera initial position value (210) in which information such as the position, angle, and relative relationship of each camera (10) is recorded, and the total angle-of-view parameter (220) that specifies the shooting range. Overlap ratio (230) that specifies the overlap ratio of images obtained from cameras, total angle of view parameter (220), focal length data (240) calculated from the overlap ratio (230), how much each camera Camera rotation amount data (250) indicating whether to rotate, camera movement amount data (260) indicating camera movement amounts for making the optical centers of all cameras (10) closest to each other are included.

  The main storage device (100) includes a camera field angle calculation unit (110) that calculates the field angle of each camera (10) based on the total field angle parameter (220) and the overlap ratio (230), and the calculation camera A focal length calculation unit (120) that calculates a focal length (240) from an angle of view, a camera rotation that calculates a rotation amount of each camera (10) from the calculated focal length data (240) and a camera posture initial value (210) An amount calculation unit (130) and a camera movement amount calculation unit (140) for calculating a movement amount of each camera (10) from the calculated camera rotation amount data (250) and the camera posture initial value (210) are included.

  In the calculation of the camera angle of view, the total angle of view parameter (220) is θ, the overlap ratio (230) is α, the number of cameras (10) is n, and the optical center of each camera (10) is almost at the same position. Assuming that there is, the angle of view φ of each camera is approximated by Equation 1. Depending on the calculated camera angle of view, the angle formed with the adjacent camera is also determined, but it is the same value φ as the camera angle of view.

  FIG. 4 is a flowchart showing the flow of processing in the present embodiment.

  As input parameters for determining the camera posture, there are a total angle-of-view parameter (220) and an overlap ratio (230). Depending on the number of cameras and the viewing angle limit of the cameras themselves, a camera configuration that satisfies both parameters may not be possible. For this reason, it is necessary to prioritize either parameter. Therefore, first, in step 1000, a priority parameter is selected. Select whether to give priority to the total angle of view or to give priority to the overlap ratio.

  If priority is given to the total angle of view, in step 1100, the total angle of view of the shooting range is input. In step 1110, it is determined whether or not the value is valid. If the entered value exceeds the maximum total viewing angle calculated from the number of cameras and the viewing angle of the camera itself, or if it is less than the camera viewing angle per camera, the value is considered invalid. Return to step 1100. In step 1100, a value that falls within the valid range is input again.

  When a valid value is input, in step 1120, the effective overlap ratio is calculated based on the total angle of view input in step 1100 and the focal length adjustment range of the camera itself.In step 1130, Enter a numeric value within the valid overlap ratio range.

  If the priority parameter is set to the overlap ratio in step 1000, the overlap ratio is input in step 1200. In step 1210, the maximum total field angle is calculated based on the input overlap ratio value, the number of cameras and the focal length adjustment range, and in step 1220, a numerical value within the effective total field angle range is input. .

  In step 1300, the angle of view per camera is calculated based on the input total angle-of-view parameter and the overlap ratio, and in step 1400, the focal length is calculated based on the camera angle of view.

  In step 1500, the camera rotation amount for satisfying the requested total field angle parameter and the overlap ratio is calculated for each camera.

  Finally, in step 1600, the movement distance for calculating the viewpoint positions of all the cameras as close as possible is calculated.

  FIG. 5 is a diagram showing a photographing range in the basic configuration in the present embodiment.

  The example of a basic composition in this embodiment is shown. The basic configuration is a configuration in which the total angle of view is set to 100 degrees, the overlap ratio is set to 20%, and the optical centers of the cameras are located at substantially the same position. In this configuration, the optical center of the camera and the angle formed with the adjacent camera are stored as the camera posture initial value (210).

  Here, the basic configuration is not limited to the total angle of view and the overlap ratio. The focal length of the camera in the basic configuration is preferably an intermediate value in the operating area. This is because adjustment is easy when the focal length is maximum and minimum. Further, the reason for installing the cameras so that the optical centers are as close as possible is to reduce inconsistencies at the time of joining the captured images. Based on this basic configuration, you can rotate the camera to the right, rotate it to the left, or move the camera to reduce the viewpoint distance while maintaining the rotation angle. Take a photo of Examples of camera configuration changes are shown in FIGS. 6, 7, 8, and 9. Fig. 6 shows the viewing angle and total shooting range of each camera after rotation adjustment when the total angle of view is 45 degrees and the overlap ratio is 20%, and Fig. 7 shows the total angle of view is 45 degrees and the overlap ratio is 20%. 8 shows the viewing angle and total shooting range of each camera after rotation and movement adjustment, and FIG. 8 shows the viewing angle of each camera after rotation adjustment when the total angle of view is 150 degrees and the overlap ratio is 20%. FIG. 9 shows the viewing angle and the total shooting range of each camera after rotation and movement adjustment when the total field angle is 150 degrees and the overlap ratio is 20%. By changing the focal length of the camera according to the total angle of view and the overlap ratio, even when the total angle of view is reduced, there is no useless overlap, and the number of captured pixels can be used effectively to the maximum. It becomes possible.

  FIG. 10 is a diagram showing an internal configuration in the second embodiment.

  In the second embodiment, in addition to the wide viewing angle image capturing of the first embodiment, a function for displaying and processing captured images is added. In addition to the configuration of FIG. 2, a photographed image deformation unit that receives a photographed image by a camera (10-a, 10-b, 10-c, 10, d, 10-e) and performs deformation for joining the images together (150) A panoramic image creation unit (160) for creating a panoramic image by stitching together the deformed images, and a panoramic image display unit (170) for displaying the created panoramic image on the display (90). included.

  FIG. 11 is a diagram showing a system configuration in the second embodiment.

  The main storage device (100) configured as shown in FIG. 3 is added with a photographed image deformation unit (150), a panorama image creation unit (160), and a panorama image display unit (170), and each camera is added to the auxiliary storage device (200). (10-a, 10-b, 10-c, 10-d, 10-e) captured image data (270) for storing images captured, and the captured image data (270) to the captured image deformation unit ( 150), and the panoramic image data (280) created by connecting the deformed image data by the panoramic image creating unit (160) is added.

1 is an apparatus configuration diagram of a first embodiment. FIG. FIG. 3 is an internal configuration diagram of the first embodiment. 1 is a system configuration diagram of a first embodiment. FIG. 3 is a flowchart showing a flow of processing in the main storage device of the first embodiment. FIG. 3 is a shooting range diagram in the basic configuration of the first embodiment. FIG. 1 is a camera posture change diagram 1 (rotation) of the first embodiment. FIG. 1 is a camera posture change diagram 1 (rotation / movement) of the first embodiment. FIG. 3 is a camera posture change diagram 2 (rotation) of the first embodiment. FIG. 3 is a camera posture change diagram 2 (rotation / movement) of the first embodiment. It is an internal block diagram of 2nd Embodiment. It is a system configuration figure of a 2nd embodiment.

Explanation of symbols

10 ... Camera, 20 ... Focal distance control device, 30 ... Rotary movement device, 40 ... Base plate, 50 ... Server PC, 60 ... CPU, 70 ... Input device, 80 ... Bus, 90 ... Display, 100 ... Main memory, 110 ... Camera angle of view calculator, 120 ... Focal distance calculator, 130 ... Rotation amount calculator, 140 ...・ Moving amount calculation unit, 150... Shot image deformation unit, 160 .. panorama image creation unit, 170 .. panorama image display unit, 200... Auxiliary storage device, 210. 220 ... total angle of view parameters, 230 ... overlap ratio, 240 ... focal length data, 250 ... camera rotation amount data, 260 ... camera movement amount data, 270 ... photographed image data, 280 ... Panorama image data

Claims (1)

In a wide viewing angle video imaging device composed of a plurality of cameras,
Each of the cameras includes a focal length control device,
Each of the cameras is mounted on a rotational movement device,
Using the total field angle parameter indicating the shooting range and the overlap ratio of adjacent shot images as inputs,
Means for calculating the angle-of-view and orientation parameters of each camera from the input total angle-of-view parameter and the input overlap ratio;
A focal length calculating means for calculating a focal length of the camera from the calculated angle of view of the camera;
Camera rotation amount calculation means for calculating the rotation amount of each camera from the calculated focal length and camera posture initial value;
By providing camera movement amount calculation means for calculating the movement amount of each camera from the calculated camera rotation amount and the camera initial position,
Wide viewing angle video camera that can control the position and orientation of the camera.

Images